-
Notifications
You must be signed in to change notification settings - Fork 184
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add Documentation for Conditional Dataset Scheduling with dag-factory #367
Add Documentation for Conditional Dataset Scheduling with dag-factory #367
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #367 +/- ##
=======================================
Coverage 94.01% 94.01%
=======================================
Files 11 11
Lines 886 886
=======================================
Hits 833 833
Misses 53 53 ☔ View full report in Codecov by Sentry. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great, thanks a lot, @ErickSeo , for follow up on the documentation!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM!
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We want to clean REAME file. I know the dataset feature we have not documented yet but I'm wondering if it makes sense to move this changes in /docs folder so that would be appear in our main docs: https://astronomer.github.io/dag-factory/latest/ ?
I'm going to move the documentation from the |
Yes, if you can do this, it would be great. |
Closes: #361
This PR adds a comprehensive guide on conditional dataset scheduling using dag-factory (v0.22.0+) and Airflow 2.9+. Key updates include:
Explanation of conditional dataset scheduling and its use cases.
Requirements for using the feature.
Examples demonstrating configurations with both string and YAML syntax.
Visual diagrams illustrating dataset condition logic.
This documentation is intended to help users understand and implement conditional dataset scheduling effectively in their workflows.