<python-cryptography>: imports that require symbols in _openssl fail

[brucealthompson]

Maintainer: Jeffery To jeffery.to@gmail.com, Alexandru Ardelean ardeleanalex@gmail.com
Environment: X86, OpenWrt trunk

Description:
There are modules within the python cryptography package that require symbols to be included in the packaged _openssl.so binary which is contained in the cryptography/hazmat/bindings directory.

By default, this package is built with symbols stripped from the _openssl.so binary. When modules from the python-cryptography package which require openssl bindings are imported in python, the import fails. Here is an example of a python statement which fails using the default python-cryptography package:

from cryptography.hazmat.bindings._openssl import ffi

The resulting error is shown in the screen dump below:
root@OpenWrtTahoe90:~# python
Python 2.7.15 (default, Mar 25 2019, 09:51:50)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from cryptography.hazmat.bindings._openssl import ffi
Traceback (most recent call last):
File "", line 1, in
ImportError: Error relocating /usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/_openssl.so: SSL_get0_next_proto_negotiated: symbol not found

To get around this issue, I have tried to build the python-cryptography package with debug information and symbols not stripped. When I build the python-cryptography package in this way, the missing symbols show up as undefined.

Here is the output of nm -g on the _openssl.so binary:root@OpenWrt:/usr/lib/python2.7/site-packages/cryptography/hazmat/bindings# nm -g _open
ssl.so | grep SSL_get0
U SSL_get0_alpn_selected@@OPENSSL_1_1_0
U SSL_get0_next_proto_negotiated
U SSL_get0_param@@OPENSSL_1_1_0

These symbols can be found in /usr/lib/libssl.so.1.1 if this library is built with debugging info and symbols not stripped. Here is the output of nm -g on this file:
root@OpenWrt:/usr/lib# nm -g libssl.so.1.1 | grep SSL_get0
000000000002e350 T SSL_get0_CA_list
0000000000038140 T SSL_get0_alpn_selected
0000000000034f50 T SSL_get0_dane
0000000000034de0 T SSL_get0_dane_authority
0000000000034e70 T SSL_get0_dane_tlsa
00000000000359a0 T SSL_get0_param
000000000002e3e0 T SSL_get0_peer_CA_list
000000000003acc0 T SSL_get0_peer_scts
0000000000034c40 T SSL_get0_peername
000000000003aba0 T SSL_get0_security_ex_data
000000000003ac90 T SSL_get0_verified_chain

It is not clear to me why, but even when I try the python import statement above with both the python-cryptography and libopenssl1.1 packages with symbols installed, I still get the same error message.

I would first like to understand why the openssl bindings in the python-cryptography package are not resolved even when both the python-cryptography and libopenssl1.1 packages with symbols are installed.

Once this issue is resolved, I think the python-cryptography needs to include a build option to enable the symbols in the _openssl.so binary of the python-cryptography package to be included. This build option is required for full python functionality from the python-cryptography package.

[neheb]

NPN was disabled by default a few months back. If python-cryptography really implements this, it should be ifdef'd out.

[jefferyto]

They have an open PR for removing NPN (pyca/cryptography#4765), seems to be blocked on an issue with Twisted's test suite.

[brucealthompson]

I brought up 2 different issues in my bug report above.

The first issue is that the the python-cryptography package is built with symbols stripped from the _openssl.so binary. I fixed this in the python-cryptography Makefile by adding the following lines:
RSTRIP:=:
STRIP:=:

The second issue is that the entry point SSL_get0_next_proto_negotiated() is not included in libssl by default. This is entry point is part of application layer protocol negotiation (ALPN) functionality in libssl. ALPN functionality can be included in libssl by including the OPENSSL_WITH_NPN flag when building it. I fixed this issue in the python-cryptography Makefile by modifying the following line:
DEPENDS:=+libopenssl >> DEPENDS:=+libopenssl @OPENSSL_WITH_NPN

The updated Makefile is attached. Since I am not the maintainer for this package, I would appreciate if the maintainers could review the changes and check them in if they are correct.
Makefile.txt

[neheb]

As @jefferyto said, upstream is working on getting rid of this.

To fix this, the pull request would either need to be patched here or wait until upstream fixes this.

An @ dependency like this is unwelcome. There have been issues regarding this in the past.

[jefferyto]

The first issue is that the the python-cryptography package is built with symbols stripped from the _openssl.so binary.

This is not an issue. Symbols are suppose to be stripped. OpenWrt is a project primarily for devices with limited storage.

The second issue is that the entry point SSL_get0_next_proto_negotiated() is not included in libssl by default.

This is not an issue. Disabling NPN does not disable ALPN, in fact ALPN replaces NPN.

The issue is that cryptography (the Python project) hasn't been updated to work with ALPN only.

I would appreciate if the maintainers could review the changes and check them in if they are correct.

I appreciate you taking the time to open this issue. Honestly I had already forgotten this was still an issue. @neheb worked on getting this fixed a few months ago (pyca/cryptography#4760).

I know your fixes work for you, but the real issue is in upstream. Getting it fixed there is the real solution.

[neheb]

Closing. This is probably not relevant anymore.

[brucealthompson]

I think the issue is still relevant. I send a private message to the package maintainers but never received a response from them. I would appreciate if we could leave this open until I receive a response to the question below:

Brucet>> The first issue is that the the python-cryptography package is built with symbols stripped from the _openssl.so binary.

JeffreyTo>>This is not an issue. Symbols are suppose to be stripped. OpenWrt is a project primarily for devices with limited storage.

I have a question in this regard. When a python module has c function bindings, doesn’t the python import statement require symbols in the c function objects in order to perform the dynamic binding? Remember the original error I got was the following:

from cryptography.hazmat.bindings._openssl import ffi
Traceback (most recent call last):
File "", line 1, in
ImportError: Error relocating /usr/lib/python2.7/site-packages/cryptography/hazmat/bindings/_openssl.so: SSL_get0_next_proto_negotiated: symbol not found
I think this is because the “_openssl.so” file in the python-cryptography package did not include any symbols.

Because of the above, I believe the general policy of stripping symbols should not hold for python modules that have c function bindings.

Have I missed something here??

[neheb]

Just tested it. I will send a pull request.