Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generalise pyinstaller spec paths #2281

Merged
merged 1 commit into from
Oct 26, 2022

Conversation

llimeht
Copy link
Contributor

@llimeht llimeht commented Oct 26, 2022

The pyinstaller spec file sometimes needs to contain workarounds for packages to help pyinstaller find all the relevant imports or data files. If such introspection is still needed, we should avoid baking in platform-dependent and python-version-dependent paths.

This PR makes use of the sysconfig module from the Python stdlib that has information about where packages are located, so the details of lib vs Lib, site-packages vs dist-packages, python3.8 vs python3.10,… all go away. This last one is very important as it means we can create installer artifacts with multiple different Python versions for testing purposes, without needing to change the spec file.

I tested an earlier version of this patch with Windows and Linux when it was still touching paths for those installers. With other changes, this code path is currently only used on macos which I can't test.

There's every chance that the actual code for jedi and zmq for Darwin can also be removed; that's a problem for a different PR and for a mac user to test. In any case, the stub of loading and using sysconfig may as well stay for the time being since it will be needed some time in the future, no doubt.

Copy link
Member

@rozyczko rozyczko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Querying core module location is a good check but it is probably better to make a check which does not require specific module name.


block_cipher = None
PYTHON_LOC = sys.exec_prefix
PYTHON_PACKAGES = sysconfig.get_path('platlib')
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Querying a random package is a bit iffy, I guess, even if it is a core package.
It seems that site.getsitepackages() results are platform dependent, so probably distutils has the best query for site_packages location

from distutils.sysconfig import get_python_lib
PYTHON_PACKAGES  = get_python_lib()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just to clarify, 'platlib' isn't asking for the location of something stdlib, it's asking for the compilation/configuration option for the interpreter for where modules get installed..

From the sysconfig docs:

platlib: directory for site-specific, platform-specific files.

For example:

>>> sysconfig.get_paths()
{'stdlib': '/usr/lib/python3.9',
 'platstdlib': '/usr/lib/python3.9',
 'purelib': '/usr/lib/python3.9/site-packages',
 'platlib': '/usr/lib/python3.9/site-packages',
 'include': '/usr/include/python3.9',
 'platinclude': '/usr/include/python3.9',
 'scripts': '/usr/bin',
 'data': '/usr'}

We absolutely should not be using distutils. It will be removed from Python 3.12 and nothing new should be using it; it's well into the deprecation phase of removing all uses of distutils.

@wpotrzebowski wpotrzebowski merged commit 3c34c04 into SasView:main Oct 26, 2022
@llimeht llimeht deleted the tmp/installer-paths branch October 26, 2022 22:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants