We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Create a "clean-up" batch job that deletes DatasetArtifacts + DatasetVersions for an input List of DatasetVersionIds.
See data model in orm.py and business logic entity objects in entities.py
Leverage existing functions for artifact/DB clean-up such as: business layer function to delete dataset version artifacts from S3
persistence layer function to delete dataset version + associated dataset artifact rows from the DB
Other batch jobs created in data-portal before: Publish revisions: code and infra Dataset metadata update: code and infra
Memory / vcpu requirements should be reasonably fine-tuned during testing in rdev environment.
The text was updated successfully, but these errors were encountered:
Note to @lvreynoso: @Bento007 has suggested looking into using an aws lambda for this instead of a batch job; I think its worth considering
Sorry, something went wrong.
lvreynoso
No branches or pull requests
Create a "clean-up" batch job that deletes DatasetArtifacts + DatasetVersions for an input List of DatasetVersionIds.
See data model in orm.py and business logic entity objects in entities.py
Leverage existing functions for artifact/DB clean-up such as:
business layer function to delete dataset version artifacts from S3
persistence layer function to delete dataset version + associated dataset artifact rows from the DB
Other batch jobs created in data-portal before:
Publish revisions: code and infra
Dataset metadata update: code and infra
Memory / vcpu requirements should be reasonably fine-tuned during testing in rdev environment.
The text was updated successfully, but these errors were encountered: