Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add missing execution-keyed entities for custom filters #6172

Merged
merged 1 commit into from
Jan 15, 2025

Conversation

katrogan
Copy link
Contributor

@katrogan katrogan commented Jan 15, 2025

Why are the changes needed?

Bug introduced in #5935

Entities in this list have execution-key as part of their primary key. Execution key columns are all prefixed with "execution_" under the hood, so we need to prepend the column values with this for DB queries to work.

What changes were proposed in this pull request?

See PR title

How was this patch tested?

Deployed to my sandbox and ran the waiting_for_external_inputs workflow on sandbox.

 pyflyte run --remote examples/advanced_composition/advanced_composition/waiting_for_external_inputs.py reporting_with_approval_wf --data '[1.0,2.0,3.0]'
/Users/katrina/.venv/union/lib/python3.12/site-packages/airflow/configuration.py:859 FutureWarning: section/key [core/sql_alchemy_conn] has been deprecated, you should use[database/sql_alchemy_conn] instead. Please update your `conf.get*` call to use the new name
Running Execution on Remote.
0:00:00 Running execution on remote.
[✔] Go to http://localhost:30080/console/projects/flytesnacks/domains/development/executions/ap8fz45qg24mcqnmcjjv to see execution in the console.

Then ran this script

from flytekit.remote import FlyteRemote
from flytekit.configuration import Config, PlatformConfig


def list_signals():
    # Step 1: Configure FlyteRemote

    remote = FlyteRemote(
        config=Config.for_sandbox(),
        default_project="flytesnacks",
        default_domain="development",
    )


    signals = remote.list_signals(execution_name="ap8fz45qg24mcqnmcjjv", project="flytesnacks", domain="development")
    print(f"signals fetched :: {signals}")

if __name__ == "__main__":
    list_signals()

and ensured it succeeded:

signals fetched :: [id {
  signal_id: "title"
  execution_id {
    project: "flytesnacks"
    domain: "development"
    name: "ap8fz45qg24mcqnmcjjv"
  }
}
type {
  simple: STRING
}
]

Process finished with exit code 0

Labels

Please add one or more of the following labels to categorize your PR:

  • added: For new features.
  • changed: For changes in existing functionality.
  • deprecated: For soon-to-be-removed features.
  • removed: For features being removed.
  • fixed: For any bug fixed.
  • security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

Summary by Bito

This PR addresses a bug in execution-keyed entities handling by adding NodeExecutionEvent and Signal to the execution identifier entities map. The SQL query column names in signal repository tests have been updated with 'execution_' prefix for proper database querying. These changes ensure correct handling of execution-keyed entities in database operations.

Unit tests added: True

Estimated effort to review (1-5, lower is better): 1

Signed-off-by: Katrina Rogan <katroganGH@gmail.com>
@katrogan katrogan added the fixed For any bug fixes label Jan 15, 2025
@katrogan katrogan requested a review from eapolinario January 15, 2025 15:53
@flyte-bot
Copy link
Collaborator

flyte-bot commented Jan 15, 2025

Code Review Agent Run #c6ef29

Actionable Suggestions - 1
  • flyteadmin/pkg/repositories/gormimpl/signal_repo_test.go - 1
Review Details
  • Files reviewed - 2 · Commit Range: 0e9387c..0e9387c
    • flyteadmin/pkg/common/filters.go
    • flyteadmin/pkg/repositories/gormimpl/signal_repo_test.go
  • Files skipped - 0
  • Tools
    • Golangci-lint (Linter) - ✖︎ Failed
    • Whispers (Secret Scanner) - ✔︎ Successful
    • Detect-secrets (Secret Scanner) - ✔︎ Successful

AI Code Review powered by Bito Logo

@eapolinario eapolinario enabled auto-merge (squash) January 15, 2025 15:54
@flyte-bot
Copy link
Collaborator

Changelist by Bito

This pull request implements the following key changes.

Key Change Files Impacted
Bug Fix - Fix Entity Mapping for Execution-Keyed Entities

filters.go - Added missing execution-keyed entities (NodeExecutionEvent and Signal) to the map

signal_repo_test.go - Updated SQL query to use correct execution-prefixed column names

@@ -138,7 +138,7 @@ func TestListSignals(t *testing.T) {
signalModels := []map[string]interface{}{toSignalMap(*signalModel)}
mockSelectQuery := GlobalMock.NewMock()
mockSelectQuery.WithQuery(
`SELECT * FROM "signals" WHERE project = $1 AND domain = $2 AND name = $3 LIMIT 20`).WithReply(signalModels)
`SELECT * FROM "signals" WHERE execution_project = $1 AND execution_domain = $2 AND execution_name = $3 LIMIT 20`).WithReply(signalModels)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider updating test filter names

The SQL query column names have been updated from project, domain, name to execution_project, execution_domain, execution_name. Consider updating the corresponding filter names in the test case to match the new column names for consistency.

Code suggestion
Check the AI-generated fix before applying
 -			getEqualityFilter(common.Signal, "project", project),
 -			getEqualityFilter(common.Signal, "domain", domain),
 -			getEqualityFilter(common.Signal, "name", name),
 +			getEqualityFilter(common.Signal, "execution_project", project),
 +			getEqualityFilter(common.Signal, "execution_domain", domain),
 +			getEqualityFilter(common.Signal, "execution_name", name),
 		},

Code Review Run #c6ef29


Is this a valid issue, or was it incorrectly flagged by the Agent?

  • it was incorrectly flagged

@eapolinario eapolinario merged commit f5e016f into master Jan 15, 2025
51 checks passed
@eapolinario eapolinario deleted the signals-execution-entity branch January 15, 2025 16:09
Copy link

codecov bot commented Jan 15, 2025

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 37.05%. Comparing base (91d0383) to head (0e9387c).
Report is 1 commits behind head on master.

Additional details and impacted files
@@           Coverage Diff           @@
##           master    #6172   +/-   ##
=======================================
  Coverage   37.05%   37.05%           
=======================================
  Files        1318     1318           
  Lines      132606   132606           
=======================================
  Hits        49136    49136           
  Misses      79220    79220           
  Partials     4250     4250           
Flag Coverage Δ
unittests-datacatalog 51.58% <ø> (ø)
unittests-flyteadmin 54.33% <ø> (ø)
unittests-flytecopilot 30.99% <ø> (ø)
unittests-flytectl 62.29% <ø> (ø)
unittests-flyteidl 7.23% <ø> (ø)
unittests-flyteplugins 53.85% <ø> (ø)
unittests-flytepropeller 42.69% <ø> (ø)
unittests-flytestdlib 55.13% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
fixed For any bug fixes
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants