Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NodeMetadata.Config property bag and expose ClusterAssignment in LaunchPlan #6173

Conversation

EngHabu
Copy link
Contributor

@EngHabu EngHabu commented Jan 15, 2025

Why are the changes needed?

This allows plugins to pass execution instructions to the backend plugin without having to introduce new protos..

What changes were proposed in this pull request?

Allow passing cluster_assignment in Workflow NodeMetadata and LaunchPlan.

How was this patch tested?

Deployed to an internal tenant and verified the bits are passed correctly.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Summary by Bito

This PR introduces a new config property bag in NodeMetadata enabling plugins to pass execution instructions without new proto definitions, and adds support for cluster_assignment in LaunchPlan specifications. The changes span across multiple language implementations including Go, TypeScript, Python, and Rust, enhancing the flexibility of plugin configuration and execution control.

Unit tests added: False

Estimated effort to review (1-5, lower is better): 5

…LaunchPlan

Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
@flyte-bot
Copy link
Collaborator

Code Review Agent Run Status

  • Limitations and other issues: ❌ Failure - The AI Code Review Agent skipped reviewing this change because it is configured to exclude certain pull requests based on the source/target branch or the pull request status. You can change the settings here, or contact the agent instance creator at eduardo@union.ai.

Copy link

codecov bot commented Jan 15, 2025

Codecov Report

Attention: Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.

Project coverage is 33.25%. Comparing base (e0833b7) to head (bf93770).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
...lyteidl/gen/pb-go/flyteidl/admin/launch_plan.pb.go 16.66% 5 Missing ⚠️
flyteidl/gen/pb-go/flyteidl/core/workflow.pb.go 16.66% 5 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #6173      +/-   ##
==========================================
- Coverage   37.05%   33.25%   -3.81%     
==========================================
  Files        1318     1020     -298     
  Lines      132627   108679   -23948     
==========================================
- Hits        49151    36138   -13013     
+ Misses      79226    69344    -9882     
+ Partials     4250     3197    -1053     
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin ?
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.23% <16.66%> (+<0.01%) ⬆️
unittests-flyteplugins 53.85% <ø> (ø)
unittests-flytepropeller 42.69% <ø> (ø)
unittests-flytestdlib 55.27% <ø> (-0.02%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@EngHabu EngHabu closed this Jan 16, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants