poetry add @latest with markers fails with error

[chargr]

Description

running poetry add using at @latest with python markers no longer works.

seems related to #10184 and/or #10185 ?

this command

$ poetry add --lock 'poetry@latest ; platform_machine!="aarch64"'

fails with

Path /home/chargreaves/tmp/testprj/latest for poetry does not exist
Path /home/chargreaves/tmp/testprj/latest for poetry does not exist

Updating dependencies
Resolving dependencies... (0.0s)

if you remove the space between the latest and semicolon, it fails differently.

$ poetry add --lock 'poetry@latest; platform_machine!="aarch64"'

Could not parse version constraint: latest;

but not combining them works fine

$ poetry add --lock 'poetry ; platform_machine!="aarch64"'

works

and

$ poetry add --lock 'poetry@latest'

works

Workarounds

downgrade to < 2.1.0

Poetry Installation Method

pip

Operating System

CentOS 7

Poetry Version

2.1.1

Poetry Configuration

$ poetry config --list
cache-dir = "/home/chargreaves/.cache/pypoetry"
data-dir = "/home/chargreaves/.local/share/pypoetry"
installer.max-workers = null
installer.no-binary = null
installer.only-binary = null
installer.parallel = true
installer.re-resolve = true
keyring.enabled = true
python.installation-dir = "{data-dir}/python"  # /home/chargreaves/.local/share/pypoetry/python
requests.max-retries = 0
solver.lazy-wheel = true
system-git-client = false
virtualenvs.create = true
virtualenvs.in-project = null
virtualenvs.options.always-copy = false
virtualenvs.options.no-pip = false
virtualenvs.options.system-site-packages = false
virtualenvs.path = "{cache-dir}/virtualenvs"  # /home/chargreaves/.cache/pypoetry/virtualenvs
virtualenvs.prompt = "{project_name}-py{python_version}"
virtualenvs.use-poetry-python = false

Python Sysconfig

No response

Example pyproject.toml

empty/simple project


poetry init --name=testprj -q

Poetry Runtime Logs

$ poetry add -vvv --lock 'poetry@latest ; platform_machine!="aarch64"'
Loading configuration file /home/chargreaves/.config/pypoetry/config.toml
Using virtualenv: /home/scratch.chargreaves_mis/projects/utils-python-build/venv
Checking keyring availability: Checking if keyring is available
[keyring:keyring.backend] Loading KWallet
[keyring:keyring.backend] Loading SecretService
[keyring:keyring.backend] Loading Windows
[keyring:keyring.backend] Loading chainer
[keyring:keyring.backend] Loading libsecret
[keyring:keyring.backend] Loading macOS
Backend 'fail Keyring' is not suitable
No valid keyring backend was found
Unavailable
Path /home/chargreaves/tmp/testprj/latest for poetry does not exist
Path /home/chargreaves/tmp/testprj/latest for poetry does not exist

Updating dependencies
Resolving dependencies...
1: fact: testprj is 0.1.0
1: derived: testprj
1: Version solving took 0.000 seconds.
1: Tried 1 solutions.

Stack trace:

19 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/application.py:327 in run
325│
326│ try:
→ 327│ exit_code = self._run(io)
328│ except BrokenPipeError:
329│ # If we are piped to another process, it may close early and send a

18 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/console/application.py:260 in _run
258│ with directory(self._working_directory):
259│ try:
→ 260│ exit_code = super()._run(io)
261│ except PoetryRuntimeError as e:
262│ io.write_error_line("")

17 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/application.py:431 in _run
429│ io.input.interactive(interactive)
430│
→ 431│ exit_code = self._run_command(command, io)
432│ self._running_command = None
433│

16 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/application.py:473 in _run_command
471│
472│ if error is not None:
→ 473│ raise error
474│
475│ return terminate_event.exit_code

15 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/application.py:457 in _run_command
455│
456│ if command_event.command_should_run():
→ 457│ exit_code = command.run(io)
458│ else:
459│ exit_code = ConsoleCommandEvent.RETURN_CODE_DISABLED

14 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/commands/base_command.py:117 in run
115│ io.input.validate()
116│
→ 117│ return self.execute(io) or 0
118│
119│ def merge_application_definition(self, merge_args: bool = True) -> None:

13 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/console/commands/installer_command.py:39 in execute
37│ def execute(self, io: IO) -> int:
38│ PoetryKeyring.preflight_check(io, self.poetry.config)
→ 39│ return super().execute(io)
40│

12 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/cleo/commands/command.py:61 in execute
59│
60│ try:
→ 61│ return self.handle()
62│ except KeyboardInterrupt:
63│ return 1

11 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/console/commands/add.py:376 in handle
374│ self.installer.whitelist([r["name"] for r in requirements])
375│
→ 376│ status = self.installer.run()
377│
378│ if status == 0 and not self.option("dry-run"):

10 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/installation/installer.py:103 in run
101│ self.verbose(True)
102│
→ 103│ return self._do_install()
104│
105│ def dry_run(self, dry_run: bool = True) -> Installer:

9 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/installation/installer.py:241 in _do_install
239│ source_root=self._env.path.joinpath("src")
240│ ):
→ 241│ solved_packages = solver.solve(
242│ use_latest=self._whitelist
243│ ).get_solved_packages()

8 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/puzzle/solver.py:87 in solve
85│ with self._progress(), self._provider.use_latest_for(use_latest or []):
86│ start = time.time()
→ 87│ packages = self._solve()
88│ # simplify markers by removing redundant information
89│ for transitive_info in packages.values():

7 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/puzzle/solver.py:168 in _solve
166│
167│ try:
→ 168│ result = resolve_version(self._package, self._provider)
169│
170│ packages = result.packages

6 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/mixology/init.py:18 in resolve_version
16│ solver = VersionSolver(root, provider)
17│
→ 18│ return solver.solve()
19│

5 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/mixology/version_solver.py:174 in solve
172│ while next is not None:
173│ self._propagate(next)
→ 174│ next = self._choose_package_version()
175│
176│ return self._result()

4 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/mixology/version_solver.py:572 in _choose_package_version
570│ package = locked
571│
→ 572│ package = self._provider.complete_package(package)
573│
574│ conflict = False

3 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/puzzle/provider.py:575 in complete_package
573│ if locked is not None and locked.package.is_same_package_as(dep):
574│ continue
→ 575│ self.search_for_direct_origin_dependency(dep)
576│
577│ dependencies = self._get_dependencies_with_overrides(_dependencies, package)

2 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/puzzle/provider.py:249 in search_for_direct_origin_dependency
247│ elif dependency.is_directory():
248│ dependency = cast("DirectoryDependency", dependency)
→ 249│ package = self._search_for_directory(dependency)
250│
251│ elif dependency.is_url():

1 /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/puzzle/provider.py:352 in _search_for_directory
350│
351│ def _search_for_directory(self, dependency: DirectoryDependency) -> Package:
→ 352│ dependency.validate(raise_error=True)
353│ package = self._direct_origin.get_package_from_directory(dependency.full_path)
354│

ValueError

Path /home/chargreaves/tmp/testprj/latest for poetry does not exist

at /home/scratch.chargreaves_mis/projects/utils-python-build/venv/lib/python3.11/site-packages/poetry/core/packages/path_dependency.py:82 in validate
78│ def validate(self, *, raise_error: bool) -> bool:
79│ if not self._validation_error:
80│ return True
81│ if raise_error:
→ 82│ raise ValueError(self._validation_error)
83│ logger.warning(self._validation_error)
84│ return False
85│
86│ @Property
(venv) [1]

[dimbleby]

Please read the docs. I see no reason to expect that the things you are doing should work?

[chargr]

In 2.1.1 this still processes correctly without @latest, it may not be in the docs, but adding markers this way has always worked. Adding markers as an arg requires each package be added one at a time, but adding them in the string allows all packages to be added at the same time with unique markers. It generates the same result as using --markers options.

[chargreaves@chargreaves-dev:testprj]$ poetry add --lock 'poetry; platform_machine!="aarch64"'
Using version ^2.1.1 for poetry

Updating dependencies
Resolving dependencies... (3.8s)

Writing lock file
(venv) [0]
[chargreaves@chargreaves-dev:testprj]$ cat pyproject.toml
[project]
name = "testprj"
version = "0.1.0"
description = ""
readme = "README.md"
requires-python = ">=3.11,<3.12"
dependencies = [
    "poetry (>=2.1.1,<3.0.0) ; platform_machine != \"aarch64\""
]


[build-system]
requires = ["poetry-core>=2.0.0,<3.0.0"]
build-backend = "poetry.core.masonry.api"

But adding an invalid marker doesnt work (and shouldnt)

$ poetry add --lock 'poetry; foo=="bar"'

Could not parse version constraint: foo=="bar"

This also only breaks when using @latest

[chargreaves@chargreaves-dev:testprj]$ poetry add --lock 'poetry@latest; platform_machine!="aarch64"'

Could not parse version constraint: latest;
(venv) [1]
[chargreaves@chargreaves-dev:testprj]$ poetry add --lock 'poetry<2; platform_machine!="aarch64"'

Updating dependencies
Resolving dependencies... (1.1s)

Writing lock file

This is why I think it's something that is expected to work, although it is not documented.

[dimbleby]

I expect the most likely outcome here is a docs update recording whatever the actual behaviour is.

At a guess that would be something like: if you want to provide a pep508 string then you can do that - but in that case it must be a real pep508 string and not include poetry specials like @latest.

Perhaps you would like to submit a pull request along those lines.

[chargr]

Sounds fair, I think I can avoid setting @latest and not mix poetry specials into the string.

I don't remember how or why I discovered that markers are also processed (probably because extras are - but that's documented). Its a very useful feature to me, intentional or not. 😂