Skip to content

Commit

Permalink
Merge pull request #10 from s-sajid-ali/cleanup_and_ci
Browse files Browse the repository at this point in the history
refactor & CI with GithubActions
  • Loading branch information
ax3l authored Apr 17, 2024
2 parents 1f334c9 + 821c966 commit af3bd95
Show file tree
Hide file tree
Showing 23 changed files with 480 additions and 251 deletions.
11 changes: 11 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Set update schedule for GitHub Actions

version: 2
updates:

- package-ecosystem: "github-actions"
directory: "/"
schedule:
# Check for updates to GitHub Actions every week
interval: "weekly"

23 changes: 23 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Python package

on: [push]

jobs:
build:

runs-on: ubuntu-latest
strategy:
matrix:
python-version: ["3.10", "3.11", "3.12"]

steps:
- uses: actions/checkout@v4
- name: Set up Python ${{ matrix.python-version }}
uses: actions/setup-python@v5
with:
python-version: ${{ matrix.python-version }}
- name: Install dependencies
run: python -m pip install .[test]
- name: Test with pytest
run: pytest

163 changes: 160 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,12 +1,169 @@
# setuptools
# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
*$py.class

# C extensions
*.so

# Distribution / packaging
.Python
build/
develop-eggs/
dist/
downloads/
eggs/
.eggs/
lib/
lib64/
parts/
sdist/
var/
wheels/
share/python-wheels/
*.egg-info/
__pycache__
*.pyc
.installed.cfg
*.egg
MANIFEST

# conda
conda_recipe/linux-64/
conda_recipe/linux-ppc64le/
conda_recipe/osx-64/
conda_recipe/win-64/

# PyInstaller
# Usually these files are written by a python script from a template
# before PyInstaller builds the exe, so as to inject date/other infos into it.
*.manifest
*.spec

# Installer logs
pip-log.txt
pip-delete-this-directory.txt

# Unit test / coverage reports
htmlcov/
.tox/
.nox/
.coverage
.coverage.*
.cache
nosetests.xml
coverage.xml
*.cover
*.py,cover
.hypothesis/
.pytest_cache/
cover/

# Translations
*.mo
*.pot

# Django stuff:
*.log
local_settings.py
db.sqlite3
db.sqlite3-journal

# Flask stuff:
instance/
.webassets-cache

# Scrapy stuff:
.scrapy

# Sphinx documentation
docs/_build/

# PyBuilder
.pybuilder/
target/

# Jupyter Notebook
.ipynb_checkpoints

# IPython
profile_default/
ipython_config.py

# pyenv
# For a library or package, you might want to ignore these files since the code is
# intended to run in multiple environments; otherwise, check them in:
# .python-version

# pipenv
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
# However, in case of collaboration, if having platform-specific dependencies or dependencies
# having no cross-platform support, pipenv may install dependencies that don't work, or not
# install all needed dependencies.
#Pipfile.lock

# poetry
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
# This is especially recommended for binary packages to ensure reproducibility, and is more
# commonly ignored for libraries.
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
#poetry.lock

# pdm
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
#pdm.lock
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
# in version control.
# https://pdm.fming.dev/#use-with-ide
.pdm.toml

# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
__pypackages__/

# Celery stuff
celerybeat-schedule
celerybeat.pid

# SageMath parsed files
*.sage.py

# Environments
.env
.venv
env/
venv/
ENV/
env.bak/
venv.bak/

# Spyder project settings
.spyderproject
.spyproject

# Rope project settings
.ropeproject

# mkdocs documentation
/site

# mypy
.mypy_cache/
.dmypy.json
dmypy.json

# ruff Python linter
.ruff_cache/

# Pyre type checker
.pyre/

# pytype static type analyzer
.pytype/

# Cython debug symbols
cython_debug/

# PyCharm
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
# be found at /~https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
# and can be added to the global gitignore or merged into this file. For a more nuclear
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
#.idea/
32 changes: 11 additions & 21 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# openPMD Updater

[![Build Status `master`](https://img.shields.io/travis/openPMD/openPMD-updater/master.svg?label=master)](https://travis-ci.com/openPMD/openPMD-updater/branches)
[![CI:ubuntu](https://github.com/openPMD/openPMD-updater/actions/workflows/ci.yml/badge.svg)](https://github.com/openPMD/openPMD-updater/actions/workflows/ci.yml)
![Supported Python Versions](https://img.shields.io/pypi/pyversions/openPMD-updater.svg)
[![License](https://img.shields.io/badge/license-ISC-blue.svg)](https://opensource.org/licenses/ISC)

Expand Down Expand Up @@ -43,10 +43,10 @@ git clone /~https://github.com/openPMD/openPMD-updater.git
cd openPMD-updater

# install dependencies
pip install -r requirements.txt
pip install .

# optional: append --user
python setup.py install
# with optional dependencies for testing
pip install .[test]
```

## Usage
Expand Down Expand Up @@ -77,26 +77,16 @@ updater.update(target_version="2.0.0", in_place=True)

### Testing

Additional packages need to be installed for the tests. To install them do:
```bash
export PYTHONPATH=$(pwd):$PYTHONPATH
pip install .[test]
```

Then one can run the tests as follows:
```bash
# all
nosetests
pytest

# specific
nosetests tests/test_backend_h5.py

# manual
python tests/test_backend_h5.py
```

*note*: this changes your `example_files/1_1_0/structure.h5` files in-place.
`git checkout` it for a reset or do a comparison:

```bash
h5diff -c example_files/1_1_0/structure.h5.bak example_files/1_1_0/structure.h5
attribute: <openPMD of </>> and <openPMD of </>>
35 differences found
Not comparable: <openPMDextension> is of class H5T_INTEGER and <openPMDextension> is of class H5T_STRING
# ...
pytest tests/test_backend_h5.py
```
Binary file removed example_files/1_1_0/structure.h5
Binary file not shown.
55 changes: 37 additions & 18 deletions openpmd_updater/Updater.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,16 @@
Authors: Axel Huebl
License: ISC
"""

import packaging.version
from openpmd_updater.backends.HDF5 import HDF5
from openpmd_updater.transforms.v2_0_0 import \
DataOrder, GridUnit, ParticleBoundary, ExtensionString, Version
from .backends.HDF5 import HDF5
from .transforms.v2_0_0 import (
DataOrder,
GridUnit,
ParticleBoundary,
ExtensionString,
Version,
)


class Updater(object):
Expand All @@ -18,22 +24,22 @@ class Updater(object):
(see: openpmd_updater.backend.IBackend) for a given file and applies a set
of transformations (see: openpmd_updater.transforms.ITransform) on it.
"""

def __init__(self, filename, verbose=False):
"""Open a file with suiteable file format backend.
"""
"""Open a file with suiteable file format backend."""
# supported file format backends
self.backends = [HDF5]

# dict of updates with each a list of transforms
# keys: "to_version" of targeted openPMD standard version
# values: ordered list of transforms
self.updates = {
"2.0.0" : [
"2.0.0": [
DataOrder.DataOrder, # must be before move of particleBoundary
GridUnit.GridUnit,
ParticleBoundary.ParticleBoundary,
ExtensionString.ExtensionString,
Version.Version # must be last
Version.Version, # must be last
]
}

Expand All @@ -44,13 +50,14 @@ def __init__(self, filename, verbose=False):

for b in self.backends:
if self.verbose:
print("[Updater] Trying file format {0}".format(type(b)))
print("[Updater] Trying file format {0}".format(b.__name__))
if b.can_handle(filename):
self.fb = b(filename)
break
if self.fb is None:
raise RuntimeError("No matching file format backend found for "
"{0}!".format(filename))
raise RuntimeError(
"No matching file format backend found for " "{0}!".format(filename)
)

def update(self, new_version="2.0.0", in_place=True):
"""Perform update to new version of the openPMD standard"""
Expand All @@ -60,8 +67,15 @@ def update(self, new_version="2.0.0", in_place=True):
# check if new version is known by the updater
update_valid = False
if new_version not in self.updates.keys():
raise RuntimeError("Only updates to openPMD standard(s) '{0}' are "
"supported!".format(" ".join(self.updates.keys())))
raise RuntimeError(
"Only updates to openPMD standard(s) '{0}' are " "supported!".format(
" ".join(self.updates.keys())
)
)

if file_version == packaging.version.parse(new_version):
print("[Updater] File is at already at the requested version!")
return

# select proper update depending on initial version
# note: multiple updates over intermediate openPMD standard releases are possible
Expand All @@ -70,18 +84,23 @@ def update(self, new_version="2.0.0", in_place=True):
if file_version <= packaging.version.parse("1.1.0"):
update_valid = True
if self.verbose:
print("[Updater] Performing update from {0} to "
"{1}".format(file_version, new_version))
print(
"[Updater] Performing update from {0} to " "{1}".format(
file_version, new_version
)
)
for t in self.updates[new_version]:
self.fb.cd(None)
next_transform = t(self.fb)
if self.verbose:
name, desc = t.name
print("[Updater] Transform {0}: "
"{1}".format(name, desc))
print("[Updater] Transform {0}: " "{1}".format(name, desc))
next_transform.transform(in_place)
file_version = packaging.version.parse(new_version)

if not update_valid:
raise RuntimeError("Unsupported openPMD standard '{0}' in file "
"'{1}':".format(file_version, self.filename))
raise RuntimeError(
"Unsupported openPMD standard '{0}' in file " "'{1}':".format(
file_version, self.filename
)
)
5 changes: 5 additions & 0 deletions openpmd_updater/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
"⏫ Update openPMD files to newer versions of the openPMD standard"

from .Updater import *
from .backends import *
from .transforms import *
Loading

0 comments on commit af3bd95

Please sign in to comment.