Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Add patches to find zlibng correctly #358

Closed
wants to merge 5 commits into from

Conversation

hmaarrfk
Copy link
Contributor

Sorry for the long patch. but this addresses a few things:

  1. It attempts to find the system installation of libz-ng correctly.
  2. It fixes blosc2.c to use libz-ng primitives instead of linking accidentally to libz

I don't think it is attempting to fall back to libz. We could do that.... but honestly, I don't think it is too sustainable.

That is your choice of course. But for conda-forge I think it is better to link to the conda-forge version of libz-ng (which I just packaged) instead of using the versioned stuff.

I believe that using libz passes all current tests, but.... just in case, I packaged libz-ng 2.0.5 and these patches were found to be useful to get it to link to the correct libz-ng instead of erroneously linking to libz.

@hmaarrfk hmaarrfk changed the title Add patches to find zlibng correctly WIP: Add patches to find zlibng correctly Sep 24, 2021
@hmaarrfk
Copy link
Contributor Author

I set this to WIP since it seems to be failing on windows presently. I'll update the patch to ensure that some windows library passes.

@hmaarrfk hmaarrfk force-pushed the findzlibng branch 3 times, most recently from a9030a7 to 92aca9b Compare September 26, 2021 12:49
@FrancescAlted
Copy link
Member

Uh, I don't like the fact that this starts using the native API for zlib-ng. It is curious because you are still using the -DZLIB_COMPAT symbol here. Frankly, I would prefer to continue using the original zlib API.

@hmaarrfk
Copy link
Contributor Author

Thanks for the feedback.

i remember that symbol clashing was a good way to cause problems. even if have different so or dll files.

I don't think i compiles zlib-ng with compatibility mode, but I think that is relatively easy to fix.

@hmaarrfk
Copy link
Contributor Author

https://stackoverflow.com/questions/22004131/is-there-symbol-conflict-when-loading-two-shared-libraries-with-a-same-symbol

above is a post that shows the potential pitfalls when symbols clash.

i would kinda rather use the direct zlib-ng specific symbols if we can, and work to maintain compatibility with zlib 1

@FrancescAlted
Copy link
Member

https://stackoverflow.com/questions/22004131/is-there-symbol-conflict-when-loading-two-shared-libraries-with-a-same-symbol

above is a post that shows the potential pitfalls when symbols clash.

i would kinda rather use the direct zlib-ng specific symbols if we can, and work to maintain compatibility with zlib 1

This is one of the cases that C-Blosc2 tries to address. By vendoring the codecs inside of the library, you are creating a kind of isolated environment that should protect you against these potential pitfalls.

What's your primary driver on removing the included codecs outside of the packaging?

@hmaarrfk
Copy link
Contributor Author

The primary motivation is packaging and licensing.

@hmaarrfk
Copy link
Contributor Author

I'm kinda surprised that you are advocating for vendoring. but you also prefer to use the old symbols.

@FrancescAlted
Copy link
Member

My question is why you want to detect external codec libs instead of not just use the vendored codecs. Just because of size or it is something else?

@hmaarrfk
Copy link
Contributor Author

I'm not sure I have anything more to add to the "static" vs "dynamically" linked debate.

This, document
/~https://github.com/conda-forge/cfep/blob/main/cfep-18.md

and the two linked references are likely better written than anything I want.

My direct goal is to be more in-line with the conda-forge packaging guidelines.

@hmaarrfk hmaarrfk force-pushed the findzlibng branch 4 times, most recently from cdbd4d4 to 129c050 Compare September 27, 2021 17:49
@hmaarrfk
Copy link
Contributor Author

@FrancescAlted I understand that you aren't so interested in using external dependencies for ZLIBNG.

However, I would appreciate your time in debugging the current builds.

Do you know how I can run the tests with gdb? What command should I run to get:

I seem to be hitting

BLOSC2_ERROR_CODEC_SUPPORT = -7,    //!< Codec not supported

for the following tests:

 The following tests FAILED:
	1648 - test_compat_blosc-1.11.1-zlib.cdata (Failed)
	1653 - test_compat_blosc-1.14.0-zlib.cdata (Failed)
Errors while running CTest
	1662 - test_compat_blosc-1.3.0-zlib.cdata (Failed)
	1666 - test_compat_blosc-1.7.0-zlib.cdata (Failed)

@hmaarrfk
Copy link
Contributor Author

Many of the other compatibility ctests pass.

@hmaarrfk
Copy link
Contributor Author

I think i have something wrong with the configuration. I've chosen to go about this a slightly different way and I'll make a new PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants