Can't add private git repo

[brno32]

• I am on the latest Poetry version.

• I have searched the issues of this repo and believe that this is not a duplicate.

• If an exception occurs when executing a command, I executed it again in debug mode (-vvv option).

• Windows 10:

• 1.1.6:

Issue

When running poetry add git+/~https://github.com/owner/repo-name.git, I get the following error:

CalledProcessError

  Command '['git', '--git-dir', 'C:/Users/brno3/AppData/Local/Temp/pypoetry-git-repo-name2cgzeklh/.git', '--work-tree', 'C:/Users/brno3/AppData/Local/Temp/pypoetry-git-repo-name-xml2cgzeklh', 'checkout', 'master']' returned non-zero exit status 1.

  at ~\.poetry\lib\poetry\utils\_compat.py:217 in run
      213│                 process.wait()
      214│                 raise
      215│             retcode = process.poll()
      216│             if check and retcode:
    → 217│                 raise CalledProcessError(
      218│                     retcode, process.args, output=stdout, stderr=stderr
      219│                 )
      220│         finally:
      221│             # None because our context manager __exit__ does not use them.

This repository is private. Running with -vvv doesn't show much else:

(venv) PS C:\Users\brno3\Repos\clients\owner\parent-repo> poetry add git+/~https://github.com/owner/repo-name.git -vvv
Using virtualenv: C:\Users\brno3\Repos\clients\owner\parent-repo\venv

Updating dependencies
Resolving dependencies...
   1: fact: parent-repo is 0.1.0
   1: derived: parent-repo
   1: Version solving took 2.102 seconds.
   1: Tried 1 solutions.

  Stack trace:

  18  ~\.poetry\lib\poetry\_vendor\py3.9\clikit\console_application.py:131 in run
       129│             parsed_args = resolved_command.args
       130│ 
     → 131│             status_code = command.handle(parsed_args, io)
       132│         except KeyboardInterrupt:
       133│             status_code = 1

  17  ~\.poetry\lib\poetry\_vendor\py3.9\clikit\api\command\command.py:120 in handle
       118│     def handle(self, args, io):  # type: (Args, IO) -> int
       119│         try:
     → 120│             status_code = self._do_handle(args, io)
       121│         except KeyboardInterrupt:
       122│             if io.is_debug():

  16  ~\.poetry\lib\poetry\_vendor\py3.9\clikit\api\command\command.py:171 in _do_handle
       169│         handler_method = self._config.handler_method
       170│ 
     → 171│         return getattr(handler, handler_method)(args, io, self)
       172│ 
       173│     def __repr__(self):  # type: () -> str

  15  ~\.poetry\lib\poetry\_vendor\py3.9\cleo\commands\command.py:92 in wrap_handle
        90│         self._command = command
        91│ 
     →  92│         return self.handle()
        93│
        94│     def handle(self):  # type: () -> Optional[int]

  14  ~\.poetry\lib\poetry\console\commands\add.py:173 in handle
       171│             self._installer.whitelist([r["name"] for r in requirements])
       172│
     → 173│             status = self._installer.run()
       174│         except BaseException:
       175│             # Using BaseException here as some exceptions, eg: KeyboardInterrupt, do not inherit from Exception

  13  ~\.poetry\lib\poetry\installation\installer.py:103 in run
       101│         local_repo = Repository()
       102│ 
     → 103│         return self._do_install(local_repo)
       104│
       105│     def dry_run(self, dry_run=True):  # type: (bool) -> Installer

  12  ~\.poetry\lib\poetry\installation\installer.py:235 in _do_install
       233│             )
       234│ 
     → 235│             ops = solver.solve(use_latest=self._whitelist)
       236│         else:
       237│             self._io.write_line("Installing dependencies from lock file")

  11  ~\.poetry\lib\poetry\puzzle\solver.py:65 in solve
        63│         with self._provider.progress():
        64│             start = time.time()
     →  65│             packages, depths = self._solve(use_latest=use_latest)
        66│             end = time.time()
        67│

  10  ~\.poetry\lib\poetry\puzzle\solver.py:233 in _solve
       231│ 
       232│         try:
     → 233│             result = resolve_version(
       234│                 self._package, self._provider, locked=locked, use_latest=use_latest
       235│             )

   9  ~\.poetry\lib\poetry\mixology\__init__.py:7 in resolve_version
       5│     solver = VersionSolver(root, provider, locked=locked, use_latest=use_latest)
       6│
     → 7│     return solver.solve()
       8│

   8  ~\.poetry\lib\poetry\mixology\version_solver.py:84 in solve
        82│             while next is not None:
        83│                 self._propagate(next)
     →  84│                 next = self._choose_package_version()
        85│
        86│             return self._result()

   7  ~\.poetry\lib\poetry\mixology\version_solver.py:400 in _choose_package_version
       398│             version = locked
       399│ 
     → 400│         version = self._provider.complete_package(version)
       401│
       402│         conflict = False

   6  ~\.poetry\lib\poetry\puzzle\provider.py:452 in complete_package
       450│                     self.search_for_file(r)
       451│                 elif r.is_vcs():
     → 452│                     self.search_for_vcs(r)
       453│                 elif r.is_url():
       454│                     self.search_for_url(r)

   5  ~\.poetry\lib\poetry\puzzle\provider.py:163 in search_for_vcs
       161│             return [self._deferred_cache[dependency]]
       162│ 
     → 163│         package = self.get_package_from_vcs(
       165│             dependency.source,

   4  ~\.poetry\lib\poetry\puzzle\provider.py:196 in get_package_from_vcs
       194│             reference = branch or tag or rev
       195│             if reference is not None:
     → 196│                 git.checkout(reference, tmp_dir)
       197│             else:
       198│                 reference = "HEAD"

   3  ~\.poetry\lib\poetry\_vendor\py3.9\poetry\core\vcs\git.py:229 in checkout
       227│         args += ["checkout", rev]
       228│
     → 229│         return self.run(*args)
       230│
       231│     def rev_parse(self, rev, folder=None):  # type: (str, Optional[Path]) -> str

   2  ~\.poetry\lib\poetry\_vendor\py3.9\poetry\core\vcs\git.py:302 in run
       300│
       301│         return decode(
     → 302│             subprocess.check_output(["git"] + list(args), stderr=subprocess.STDOUT)
       303│         ).strip()
       304│

   1  C:\Python39\lib\subprocess.py:424 in check_output
        422│         kwargs['input'] = empty
        423│
     →  424│     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
        425│                **kwargs).stdout
        426│

  CalledProcessError

  Command '['git', '--git-dir', 'C:/Users/brno3/AppData/Local/Temp/pypoetry-git-repo-name2cgzeklh/.git', '--work-tree', 'C:/Users/brno3/AppData/Local/Temp/pypoetry-git-repo-name2cgzeklh', 'checkout', 'master']' returned non-zero exit status 1.

  at ~\.poetry\lib\poetry\utils\_compat.py:217 in run
      213│                 process.wait()
      214│                 raise
      215│             retcode = process.poll()
      216│             if check and retcode:
    → 217│                 raise CalledProcessError(
      218│                     retcode, process.args, output=stdout, stderr=stderr
      219│                 )
      220│         finally:
      221│             # None because our context manager __exit__ does not use them.

[finswimmer]

Hello @brno32 ,

I guess the problem here is, that the git checkout fails because of missing credentials. Instead of using poetry add git+https://... use poetry add git+ssh://... and make sure your local git is configured to provide the correct key (Just check by git clone ssh://....).

fin swimmer

[finswimmer]

Or maybe it's a duplicate of #3366 ?

[miohtama]

I confirmed the same issue and it happened with a public repository.

  CalledProcessError

  Command '['git', 'clone', '--recurse-submodules', '/~https://github.com/miohtama/sphinx.git', '/var/folders/wg/6tjsdnt511s90hqpxtxgkndc0000gn/T/pypoetry-git-sphinxib4svzng']' returned non-zero exit status 128.

  at ~/.pyenv/versions/3.8.1/lib/python3.8/site-packages/poetry/utils/_compat.py:217 in run
      213│                 process.wait()
      214│                 raise
      215│             retcode = process.poll()
      216│             if check and retcode:
    → 217│                 raise CalledProcessError(
      218│                     retcode, process.args, output=stdout, stderr=stderr
      219│                 )
      220│         finally:
      221│             # None because our context manager __exit__ does not use them.

The reason was that my SSH key was not loaded to the agent (ssh-add ~/.ssh/my-key). Though I have no idea why git wants to have my SSH key for HTTP checkout, but after 15 years I have kind of stopped trying to understand git internals.

I figured this out by running the given Command line manually from the shell to see the full git error output.

The proper fix would be to show full error log output from git subprocess, so users can figure out why git failed.

[sr-verde]

I cannot confirm that ssh-add works. Trying to add a private repository (from a private Gitlab instance) fails anyway.

But:

• Cloning the same repository manually works
• Adding a public repository from Github works fine
• Adding the dependency manually in pyproject.toml as project = {git = "ssh://git@git.example.com/some/project.git"} and then run poetry update works fine

[sr-verde]

@finswimmer, no this repository has a master branch.

[abn]

Does #5428 make things work for this?

[fredrikaverpil]

@abn @finswimmer I've tried both poetry from latest master and 1.1.13 and I have troubles with private repos.

[abn]

@fredrikaverpil I can add prviate repos fine using master. I wonder if this issue has to do with how your ssh keys are handled?

[fredrikaverpil]

@abn do I need to handle them some special way?

I've added my key (located in ~/.ssh) in the github settings and enabled SSO for my organization (in the settings for my key). That's all.

[abn]

Nothing special. If it's a new key, make sure its added to the ssh-agent. Feel free to ping on discord.

[fredrikaverpil]

@brno32 @sr-verde were you still having issues with this?

I managed to get everything to work by using the right branch AND the right command 🤦

poetry add git+ssh://git@github.com/privateorg/privaterepo.git#main

[abn]

Closing this for now, please re-open if version on master does not work for any specific case.

[github-actions]

This issue has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.