[CDF-28553] Add lineageAnnotation support to workflow version tasks - #3208
[CDF-28553] Add lineageAnnotation support to workflow version tasks#3208Magssch wants to merge 5 commits into
Conversation
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize the Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counterproductive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. |
|
/gemini review |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces support for workflow task lineage annotations, guarded by the DATA_PRODUCTS feature flag. It defines LineageInfo and TaskLineage models, adds lineage_annotation to tasks, implements serialization stripping when the feature flag is disabled, and includes comprehensive unit tests to validate these changes. I have no feedback to provide as there are no review comments.
☂️ Code Coverage
Overall Coverage
New FilesNo new covered files... Modified Files
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3208 +/- ##
==========================================
+ Coverage 85.03% 85.17% +0.13%
==========================================
Files 466 465 -1
Lines 43830 43707 -123
==========================================
- Hits 37271 37226 -45
+ Misses 6559 6481 -78
🚀 New features to boost your workflow:
|
| DATA_PRODUCTS = FlagMetadata( | ||
| visible=False, | ||
| description="Enables support for data product resources", | ||
| visible=True, |
There was a problem hiding this comment.
Making this visible since data products is now publicly documented
2 participants
Description
Adds
lineageAnnotation(sources/targets as CDF resource URIs) to workflow version tasks, in both the YAML class and the request/response resource classes. The field is gated behind the existingdata_productsalpha flag since even though it requires its own feature flag in CDF, it's main purpose is to be used with data products. The YAML validator rejects it when the flag is off, anddump_resourcestrips it from CDF responses so it does not show up in diffs/dumps. Also adds identifiable list diffing onurifor lineage sources/targets, and finally makes thedata_productsflag visible incdf aboutsince the API itself is now publicly documented.Bump
Changelog
Added
lineageAnnotationon workflow version tasks, declaring the data sources and targets of each task. Requires thedata_productsalpha flag.