♻️ REFACTOR: Proposal for RepoPath API

[chrisjsewell]

This a proposal for an implementation of the file system path protocol (PEP 519), and mimics the pathlib.Path API.
This is exposed as Node.repo_path

In addition, the Repository is sub-classed into MutableRepoFileSystem (for use before storing) and FrozenRepoFileSystem (for use after storing)

In [1]: node = Data()

In [2]: node.repo_path.joinpath('a', 'b', 'c').write_text("hallo")

In [3]: node.repo_content_string()
a: 
a/b: 
a/b/c: d0a02a6a-ba87-4db6-b4ef-833d89ecd2c1

In [4]: node.repo_path.joinpath('a', 'b', 'c')
Out[4]: RepoPath('a/b/c', MutableRepoFileSystem())

In [5]: node.put_object_from_tree('/Users/chrisjsewell/Documents/GitHub/aiida-core-origin/.molecule')

In [6]: node.repo_content_string()
README.md: 90f76af5-f271-4a7e-8483-31591750fd21
a: 
a/b: 
a/b/c: d0a02a6a-ba87-4db6-b4ef-833d89ecd2c1
default: 
default/Dockerfile: 0acbdc57-43dc-4f6f-97e7-d40e0bfd9ad7
default/config_jenkins.yml: d96a8823-2650-4150-aa1c-305e98fed76d
default/config_local.yml: 95e695a7-4044-477b-9d3c-e1d7c38021ab
default/create_docker.yml: 56285e18-c760-420d-b9ac-87acfab1bc9c
default/files: 
default/files/polish: 
default/files/polish/__init__.py: 966bbc4d-042b-4e89-8525-d823efd8bed0
default/files/polish/cli.py: 606f9226-199a-41f7-98e7-17eaa03c41be
default/files/polish/lib: 
default/files/polish/lib/__init__.py: 237d6def-eb89-48c1-b40d-df653a2920e2
default/files/polish/lib/expression.py: ca11e38a-ea14-4894-93d0-8306e6762ee9
default/files/polish/lib/template: 
default/files/polish/lib/template/base.tpl: 027eb8b3-c916-425a-988f-58f3ebb2d5b1
default/files/polish/lib/template/workchain.tpl: 296f0ecb-1b89-4405-adda-c74b81b18da9
default/files/polish/lib/workchain.py: 72e9a6ec-647c-4209-8365-ee46c2f45a25
default/run_tests.yml: 347df5b5-07a1-4bfb-82f0-24ed6a0b8cec
default/setup_aiida.yml: 91297dfa-22ac-4ca5-8925-34bfbdd48088
default/setup_python.yml: 62bce7ed-ced7-4127-beb9-46fe0d1afb50
default/tasks: 
default/tasks/log_query_stats.yml: 35a9aaa1-06fc-4086-a2b8-1d0385b039e7
default/tasks/reset_query_stats.yml: dcb3aa11-7f34-4b34-96c3-dc60f1e11836
default/test_polish_workchains.yml: 88eaf11c-d158-4792-a175-cb2813715902

In [7]: node.store()
Out[7]: <Data: uuid: 8e63d084-3e81-4555-ba07-5e373ae274b2 (pk: 9)>

In [8]: node.repo_path.joinpath('a', 'b', 'c')
Out[8]: RepoPath('a/b/c', FrozenRepoFileSystem())

In [9]: node.repo_path.joinpath('a', 'b', 'x').write_text('hi')
---------------------------------------------------------------------------
ModificationNotAllowed                    Traceback (most recent call last)
...
ModificationNotAllowed: the node is stored and therefore the repository is immutable.
In [10]: node.repo_path.joinpath('a', 'b', 'c').read_text()
Out[10]: 'hallo'
In [11]: node.repo_path.joinpath('a', 'b', 'c').is_file()
Out[11]: True

In [12]: node.repo_path.joinpath('a', 'b', 'c').exists()
Out[12]: True

In [13]: list(node.repo_path.joinpath('default').iterdir())
Out[13]: 
[RepoPath('default/setup_python.yml', FrozenRepoFileSystem()),
 RepoPath('default/config_jenkins.yml', FrozenRepoFileSystem()),
 RepoPath('default/test_polish_workchains.yml', FrozenRepoFileSystem()),
 RepoPath('default/run_tests.yml', FrozenRepoFileSystem()),
 RepoPath('default/files', FrozenRepoFileSystem()),
 RepoPath('default/Dockerfile', FrozenRepoFileSystem()),
 RepoPath('default/setup_aiida.yml', FrozenRepoFileSystem()),
 RepoPath('default/config_local.yml', FrozenRepoFileSystem()),
 RepoPath('default/tasks', FrozenRepoFileSystem()),
 RepoPath('default/create_docker.yml', FrozenRepoFileSystem())]
In [14]: list(node.repo_path.joinpath('default').walk())
Out[14]: 
[RepoPath('default', FrozenRepoFileSystem()),
 RepoPath('default/test_polish_workchains.yml', FrozenRepoFileSystem()),
 RepoPath('default/setup_python.yml', FrozenRepoFileSystem()),
 RepoPath('default/config_jenkins.yml', FrozenRepoFileSystem()),
 RepoPath('default/Dockerfile', FrozenRepoFileSystem()),
 RepoPath('default/run_tests.yml', FrozenRepoFileSystem()),
 RepoPath('default/setup_aiida.yml', FrozenRepoFileSystem()),
 RepoPath('default/config_local.yml', FrozenRepoFileSystem()),
 RepoPath('default/create_docker.yml', FrozenRepoFileSystem()),
 RepoPath('default/tasks', FrozenRepoFileSystem()),
 RepoPath('default/tasks/log_query_stats.yml', FrozenRepoFileSystem()),
 RepoPath('default/tasks/reset_query_stats.yml', FrozenRepoFileSystem()),
 RepoPath('default/files', FrozenRepoFileSystem()),
 RepoPath('default/files/polish', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/__init__.py', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/cli.py', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/workchain.py', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/__init__.py', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/expression.py', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/template', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/template/base.tpl', FrozenRepoFileSystem()),
 RepoPath('default/files/polish/lib/template/workchain.tpl', FrozenRepoFileSystem())]

[chrisjsewell]

Just gonna casually leave this here @sphuber 😬 😆

[sphuber]

Since this has been open for a long time without activity, I think we can close this for now. Happy to discuss an improvement to the interface, but any changes will probably have to wait till v3.0 and probably a proper AEP may be in place to discuss such a significant change.

@chrisjsewell could you please copy the branch from the main repo to your fork for safekeeping so we can remove it here? Cheers