DLL load failed on Appveyor Windows

[djhoese]

I have a project that is using astropy's ci-helpers to build a conda environment on appveyor. Nothing has been changed by us in our build setup, but we are now getting a DLL load failed error message:

  File "c:\projects\satpy\satpy\writers\geotiff.py", line 30, in <module>
    from osgeo import gdal, osr
  File "C:\Python27_64\envs\test\lib\site-packages\osgeo\__init__.py", line 25, in <module>
    _gdal = swig_import_helper()
  File "C:\Python27_64\envs\test\lib\site-packages\osgeo\__init__.py", line 21, in swig_import_helper
    _mod = imp.load_module('_gdal', fp, pathname, description)
ImportError: DLL load failed: The specified module could not be found.

You can see the full job and environment here: https://ci.appveyor.com/project/pytroll/satpy/builds/20702760/job/x0mo4l3940ff4n9h?fullLog=true

As far as I can tell both libgdal, gdal, and kealib are all coming from conda-forge. Any other ideas to what has changed recently that would cause this?

[mingwandroid]

Are you activating your environment?

[djhoese]

Yes, it is done automatically by ci-helpers. You can see the path to the environment packages in the error message: C:\Python27_64\envs\test

Edit: Also, I haven't changed anything in my build scripts or CI environment files.

[mingwandroid]

Given we see: /~https://github.com/astropy/ci-helpers/blob/6ac17697e48002ec74bd8ef98bf6376a475a5059/appveyor/install-miniconda.ps1#L288

I suspect activation doesn't work right here. I don't know anything about powershell though so can't be of much more help.

[djhoese]

Hm interesting. There is the activate test line above that and I do see a couple changes over the last couple days. I'll try using an older commit of ci-helpers and see if that fixes things and if so file a bug with them. Thanks.

[gillins]

BTW we no longer support Python 2.7 on Windows (gdal won't compile on MSVC 2008)... Maybe there should be a better error message about this?

[djhoese]

Thanks @gillins. As long as there are older versions available on conda-forge until the end of 2019 I shouldn't be too affected.

[djhoese]

I went back to astropy/ci-helpers@4de3957#diff-567199671add953f07327d2790dbb56a

And still have the error. I compared the conda package versions from the last successful run from a day ago and the biggest difference I noticed is krb5 going from 1.14.6 to 1.16.2. Trying to force the version now and see if that changes anything.

[djhoese]

krb5 didn't change anything. The other big change that I don't have much control over is the old working build is showing vc9 for a ton of packages and the new ones aren't. Trying to revert sip now too since I noticed that version was different. Other differences include: curl, libpq, pbr

[djhoese]

I found some other earlier failing builds and it seems like the biggest change is the vc9 feature set. Is there anyway to ask for that specific version/feature of the packages?

@gillins I also wanted to mention that even though I made this issue about python 2.7, this linking issue is also on our 3.6 environment.

[djhoese]

FYI I was able to reproduce this on a Windows 7 VM with Python 3.6. Now to track down exactly what library/link is causing the issue.

@ocefpaf I know you've helped me with linux linking issues with gdal/rasterio in the past. Do you have any ideas on this one?

[djhoese]

Progress! I copied .../my_conda_env/Library/bin/tiff.dll to .../my_conda_env/Library/bin/libtiff.dll and things seem to be importable on my VM. Will try this hack on the appveyor environment.

[ocefpaf]

@ocefpaf I know you've helped me with linux linking issues with gdal/rasterio in the past. Do you have any ideas on this one?

Sorry but Windows is not my thing. @gillins probably knows better.

PS: this issue is probably related to #242

[djhoese]

@gillins @ocefpaf I've figured out the issue for my environments. For the last couple builds of libtiff the shared library on Windows is called tiff.dll, but gdal seems to be built with one of the older builds which uses libtiff.dll. If I add a small hack to my appveyor.yaml that copies tiff.dll to libtiff.dll then gdal imports as expected:

python -c "from osgeo import gdal, osr"

I'll file a bug with the libtiff feedstock with further details unless you can quickly reply to this letting me know that this is an expected change for libtiff.

[ocefpaf]

I'll file a bug with the libtiff feedstock with further details unless you can quickly reply to this letting me know that this is an expected change for libtiff.

I can reproduce it in #243 but the odd thing is that libtiff is not changed in ages :-/ (I'll take a closer look into that tomorrow.)

[djhoese]

Thanks for the info @ocefpaf. See conda-forge/libtiff-feedstock#35 for some things I've been looking at and some wild guesses I have.

[kmuehlbauer]

@djhoese @ocefpaf AFAIK, gdal has an internal copy of libtiff. Could it be, that the gdal build process has changed somehow, using internal libtiff copy. There have been recent changes to libtiff I read on the gdal mailing list.

[djhoese]

@kmuehlbauer The conda-forge recipe at least has depended on libtiff and geotiff for some time: /~https://github.com/conda-forge/gdal-feedstock/blob/master/recipe/meta.yaml#L47

Also note that older versions of gdal (not just the new builds/versions) like v2.2.4 also fail with the most recent builds of libtiff (non-vc9/vc14).

[djhoese]

The question is, is the libtiff.dll -> tiff.dll desired by libtiff? Is that a bug or unexpected result from a change in conda-build? If gdal was built against the new builds of libtiff (tiff.dll), would it have picked up tiff.dll? Is that a limitation of gdal's build scripts that it didn't use tiff.dll but should be able to? I guess it depends on what is considered standard practice for libraries on Windows and if the libtiff name change was desired.

[djhoese]

Another discovery, geotiff depends on libtiff and according to "Dependency Walker" it also is looking for libtiff.dll and can't find it.

[mingwandroid]

Is that a bug or unexpected result from a change in conda-build

Never say never of course but I think it very unlikely that anything in conda-build would have any bearing on this issue.

[ocefpaf]

Never say never of course but I think it very unlikely that anything in conda-build would have any bearing on this issue.

BTW, @djhoese already found the issue and it is from when I moved the recipe to be more like AR and removed a workaround we had in place for it. B/c of that believe the right "fix" is in gdal and not libtiff.