Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(stepfunctions): add support JSONata and variables #32343

Open
wants to merge 56 commits into
base: main
Choose a base branch
from

Conversation

WinterYukky
Copy link
Contributor

@WinterYukky WinterYukky commented Dec 1, 2024

Issue #32262

Closes #32262.

Reason for this change

For to use JSONata and variables of Step Function feature on AWS CDK. JSONata is new query language of Step Function, It is simple and powerful language. JSONata and variables is recommend for new state machine.

Description of changes

JSONata support

Add jsonPath() and jsonata() factory methods to state constructs. For example,

// For JSONPath
sfn.Pass.jsonPath(stack, "JSONPathPass", {
    outputPath: sfn.JsonPath.stringAt('$foo'),
});

// For JSONata
sfn.Pass.jsonata(stack, "JSONataPass", {
    outputs: {
        count: "{% $states.input.count + 1 %}"
    },
});

One option would be to simply add JSONata-specific properties to the Props of the existing State construct, but in this case, the JSONata-specific properties will be displayed to the JSONPath user. Conversely, it was thought that the development experience would deteriorate if JSONata users were shown JSONPath-specific properties. As a countermeasure, we decomposed the existing Props into JSONPath-specific properties and created jsonPath() and jsonata() factory methods to separate type suggestions for JSONPath users and JSONata users.

The existing initialization method, the constructor, is backward compatible because it accepts JSONPath and JSONata properties. However, to use this interface directly is a lot of noise. This noise is a source of confusion for SFn beginners, and I thought it was necessary to solve this problem.
image

Therefore, we use the factory methods for each query language.
※ The output property is used in the image example, but it is actually outputs. Check out this comment for the reasons for this decision.

jsonPath() does not have a JSONata-specific property outputs.
image

jsonata() does not have JSONPath-specific properties such as xxxPath.
image

Variables

Add assign to state constructs. assign can be used from either JSONata or JSONPath.

For example,

sfn.Pass.jsonata(this, 'AssignExamplePass', {
  assign: {
     count: "{% $states.input.count + 1 %}"
  },
});

Description of how you validated changes

Added unit test. Integration tests are not yet.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1 admired-contributor [Pilot] contributed between 13-24 PRs to the CDK labels Dec 1, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team December 1, 2024 16:16
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@WinterYukky WinterYukky changed the title feat(step-functions): Add support JSONata and variables feat(stepfunctions): Add support JSONata and variables Dec 1, 2024
@WinterYukky WinterYukky changed the title feat(stepfunctions): Add support JSONata and variables feat(stepfunctions): add support JSONata and variables Dec 2, 2024
Copy link

@melalawi melalawi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gave it a first pass-through. Good stuff! 😸

One thing we enforce is that a state machine whose QueryLanguage is set to JSONata will not accept any states with JSONPath as their QueryLanguage. Can this also be enforced here?

packages/aws-cdk-lib/aws-stepfunctions/lib/types.ts Outdated Show resolved Hide resolved
@WinterYukky
Copy link
Contributor Author

WinterYukky commented Dec 3, 2024

@melalawi Thank you for your reviewing😀
Yeah, I implementing to throw error when top level QueryLanguage is JSONata but state level QueryLanguage is JSONPath.

protected renderQueryLanguage(topLevelQueryLanguage?: QueryLanguage): any {
topLevelQueryLanguage = topLevelQueryLanguage ?? QueryLanguage.JSONPATH;
if (topLevelQueryLanguage === QueryLanguage.JSONATA && this.queryLanguage === QueryLanguage.JSONPATH) {
throw new Error(`'queryLanguage' can not be 'JSONPath' if set to 'JSONata' for whole state machine ${this.node.path}`);
}
const queryLanguage = topLevelQueryLanguage === QueryLanguage.JSONPATH && this.queryLanguage === QueryLanguage.JSONATA
? QueryLanguage.JSONATA : undefined;
return {
QueryLanguage: queryLanguage,
};
}

GavinZZ
GavinZZ previously requested changes Dec 5, 2024
Copy link
Contributor

@GavinZZ GavinZZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this huuge effort to enable JSONata with State Machine Chainable constructs. Appreciate the effort.

I'm still going through some of the changes, but overall it looks good and I don't see any breaking changes. I left some minor feedback on the parts that I've reviewed.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Jan 3, 2025
@gracelu0 gracelu0 reopened this Jan 3, 2025
@gracelu0 gracelu0 removed the closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. label Jan 3, 2025
@aws-cdk-automation
Copy link
Collaborator

This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error.

@aws-cdk-automation aws-cdk-automation added the closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. label Jan 4, 2025
@GavinZZ GavinZZ reopened this Jan 6, 2025
@aws-cdk-automation
Copy link
Collaborator

This PR has been deemed to be abandoned, and will be automatically closed. Please create a new PR for these changes if you think this decision has been made in error.

@gracelu0 gracelu0 reopened this Jan 7, 2025
@gracelu0 gracelu0 added pr-linter/do-not-close The PR linter will not close this PR while this label is present and removed closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. labels Jan 7, 2025
@aws-cdk-automation aws-cdk-automation dismissed their stale review January 7, 2025 17:22

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@mergify mergify bot dismissed GavinZZ’s stale review January 8, 2025 07:50

Pull request has been modified.

@WinterYukky WinterYukky marked this pull request as ready for review January 17, 2025 02:36
@aws aws unlocked this conversation Jan 17, 2025
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 1394899
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
admired-contributor [Pilot] contributed between 13-24 PRs to the CDK effort/medium Medium work item – several days of effort feature-request A feature should be added or improved. p1 pr-linter/do-not-close The PR linter will not close this PR while this label is present
Projects
None yet
Development

Successfully merging this pull request may close these issues.

stepfunctions: support variables and jsonata
7 participants