poetry sync (editable mode) doesn't handle/allow custom *.pth files

[tik-stbuehler]

Issue Kind

Brand new capability

Description

The wheel build via poetry build includes custom *.pth files, but they are not installed in editable mode, and there doesn't seem to be an option to enforce this (also couldn't find anything in the source that might do it).

Code references for editable wheel build (which is later installed to the virtualenv):

• /~https://github.com/python-poetry/poetry-core/blob/eee84d755cf3ac4573efa552cec283d4e7f8758e/src/poetry/core/masonry/builders/wheel.py#L127-L146
  • it creates a $PROJECT.pth-file linking the "editable" packages (actually their base directory, i.e. the "src" folder from the source) into the virtualenv (this works for finding *.py files, but not *.pth)
  • _build() doesn't do anything with the wheel in editable mode
  • _copy_file_scripts() only installs script files into a data/scripts subfolder.
  • dist-info / metadata is a subfolder too

Impact

I use *.pth-files to initialize the environment; this is an application and is always installed in its own virtualenv, so there is no downside, but it makes life much easier.

Workarounds

With hatch instead of poetry I can install *.pth-files using [tool.hatch.build.targets.wheel.force-include].

[Secrus]

It sounds like you just need to include the file in the wheel

[tool.poetry]
include = [
    { path = "*.pth", format = "wheel" },
]

[tik-stbuehler]

It sounds like you just need to include the file in the wheel

[tool.poetry]
include = [
    { path = "*.pth", format = "wheel" },
]

As i said: it ends up in the "normal" wheel just fine, but not in the "editable" wheel (which basically only contains the auto-created PROJECT.pth-file and some metadata).

In "editable" mode it doesn't call _copy_module(), which would normally add the "included" files to the wheel; it calls _add_pth(), which only collects (base) paths of packages to write into the PROJECT.pth file.

[Secrus]

PRs are welcome

[dimbleby]

PRs are welcome

current behaviour seems correct to me. All files are not included in editable wheels, I see no reason to make a special case of some?

[tik-stbuehler]

PRs are welcome

Before I (or somebody else) tries to implement support for this it'd be nice to know if it'd be considered (and whether it needs to be hidden behind flags / options).

current behaviour seems correct to me. All files are not included in editable wheels, I see no reason to make a special case of some?

I think "editable" basically means "can edit the source and don't need to re-install to activate the changes" (but you probably need to restart your program).

One can argue that this should mean that the editable wheel doesn't include any of the source files, but: I'd also say that "editable wheels should behave the same as normal wheels" is a reasonable requirement.

I see a few ways to go:

• don't support *.pth-files in editable installs (as it is now in poetry)
• make python look for *.pth files in all search directories
• try symlinking *.pth-files instead of copying (on platforms that support it): still no new *.pth files will be found, but changes to existing ones might get applied
• just copy the *.pth-files - neither changes, deletions or creations of/to *.pth files are applied without reinstall

As far as I know poetry doesn't really support installing "non-editable" (neither via poetry sync nor via poetry install), and pip install wouldn't (properly) support packages from multiple sources.

[dimbleby]

an editable install just means arranging that the path is such that the python code in your project is available.

That does not mean copying around other files that happen to have special meaning. IMO what you are doing is non-standard, and I would not expect tools to have special support for it.