Importing from dependency during build in __init__.py fails

[b-morgenthaler]

Hi,

Here's a behavior that I try to understand and how to properly work with:

My Python application has some imports from its default dependencies in its __init__.py. Whenever build is called (either implicit or explicit), pdm complains that the imported module is not available.

E.g.:

Installing the project as an editable package...
  ✖ Update application 1.0.0.dev0+g0800c24d.local -> None failed

See C:\Users\<blackened>\AppData\Local\pdm\pdm\Logs\pdm-install-u1if94ts.log for detailed debug log.
[BuildError]: Build backend raised error: Module 'descope' is missing, please make sure it is specified in the 'build-system.requires' section. If it is not possible, add it to the project and use '--no-isolation' option. 
  File "<blackened>\__init__.py", line 1, in <module>
    from descope import DescopeClient
ModuleNotFoundError: No module named 'descope'

Note: descope is just a random library that I added as default dependency to proof my point.

I understand the error message but I would like to understand also

• the reason why it only happens if dependencies are import in the __init__,py and not in any other module
• if it is really wanted to duplicate the dependency for the build-system if dependencies are imported in __init__,py (giving up the isolation is something I would not want to do)
• if there's another configuration that I am missing to make this work without duplicating the dependency

Thanks in advance.

[frostming]

why would it import your package during build? that doesn't seem correct. you may be doing things wrong but I can't guess unless you provide more info or code

[b-morgenthaler]

I am not importing the package I am about to build, I am importing a module from a dependency.

Here's the example src layout:

repository/
├─ src/
│  ├─ application/
│  │  ├─ __init__.py
│  │  ├─ some_other_module.py
│  │  ├─ ...
├─ pyproject.toml

The pyproject.toml defines a default dependency (excerpt below):

[project]
name = "application"
version = "1.0.0"
description = "Some example application"
authors = [
    {name = "b-morgenthaler", email = "some_email@company.org"},
]
dependencies = [
    "descope" # note that this is just a random dependency - could be anything
]
requires-python = ">=3.10.14"

The __init__.py from repository/src/application looks like this:

from descope import DescopeClient

When a pdm command is used to build this application (or any command that also builds implicitly, like pdm sync -r), the aforementioned error shows.

This can be solved by duplicating the default dependency descope to the build-system, with

[build-system]
requires = ["pdm-backend", "descope"]
build-backend = "pdm.backend"

I would like to understand why this duplication is needed, only if a module is imported from a dependency in __init__.py and not in other modules (like some_other_module.py from my example above) and if this can be done without duplication and still having isolation.

EDIT: To extend the example - since you asked why you would import my own package:

Let's assume some_other_module.py looks like this:

from descope import DescopeClient


class SomeOtherModule:

   def __init__(self) -> None:
       pass

and assume __init__.py imports like this:

from .some_other_module import SomeOtherModule

The same error happens.

EDIT2: If I remove the import from the __init__.py in the latter example, the duplicated dependency for build-system is not needed anymore.

To give you some context on the latter: this is a common practise for Robot Framework keywork libraries written in Python. See for example the robotframework-browser library

[frostming]

I can't reproduce, maybe there is some special config breaking this that you are not aware of.

Can you prove an example repo to reproduce this? Maybe try it in a clean environment.

[pawamoy]

My first intuition was that a custom build hook was used, or that the version was imported from __init__.py or something, but it doesn't look like it. +1 on providing a reproducible example.

[b-morgenthaler]

My bad, I wasn't even aware of this detail until @pawamoy mentioned it: using dynamic version (in my case a getter) makes the difference. My example from above, indeed, does not reproduce the error. The example has to be extended like this (I add an archive with the source for your convenience).

pdm_build_test.zip

BTW: I reproduced it with pdm version 2.16.1 and 2.17.2, both with pdm-backend version 2.3.3

repository/
├─ src/
│  ├─ pdm_build_test/
│  │  ├─ __init__.py
│  │  ├─ some_other_module.py
│  │  ├─ version.py
├─ pyproject.toml

[project]
name = "pdm_build_test"
dynamic = ["version"]
description = "Default template for PDM package"
authors = [
    {name = "b-morgenthaler", email = "some_email@some_company.org"},
]
dependencies = [
    "descope>=1.6.6", # note that this is just a random dependency - could be anything
]
requires-python = ">=3.10.14"
readme = "README.md"
license = {text = "MIT"}

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

[tool.pdm]
distribution = true

[tool.pdm.version]
source = "call"
getter = "src.pdm_build_test.version:get_package_version"

[frostming]

so that is expected. calling a function from your package imports the __init__ module

[b-morgenthaler]

Ok. Thanks for the confirmation. Bottom line: there is no other way than to duplicate the dependency for the build-system if isolation shall not be sacrificed - or using a static version string, correct?

[frostming]

You can refactor the get version function out of your package to make it standalone

[b-morgenthaler]

Yes, thanks, that's an idea I am going to follow.

[pawamoy]

You could also extract it statically (no code execution), see https://backend.pdm-project.org/metadata/#read-from-a-static-string-in-the-given-file-path.

[b-morgenthaler]

yes, and thanks for the suggestion. That is actually what the application does but for the build pipeline, I need a more detailed version (pre-releases, dev etc.) depending on the current state of the application development life-cycle. So the only solution that worked in all scenarios where a dedicated get_version getter that I only use when building the application.