Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ESQL: Add interfaces to distribute the post-analysis verification #119798

Merged
merged 7 commits into from
Jan 10, 2025

Conversation

bpintea
Copy link
Contributor

@bpintea bpintea commented Jan 8, 2025

This adds two interfaces, PostAnalysisPlanVerificationAware and PostAnalysisVerificationAware, that allow an expression, plan or even command to perform post-analysis verifications "locally", vs. having them centralised in the core verifier.

This adds a PostAnalysisAware interface that allows an expression, plan
or even command to perform post-analysis verifications "locally", vs.
having them centralized in the core verifier.
@elasticsearchmachine
Copy link
Collaborator

Hi @bpintea, I've created a changelog YAML for you.

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - this looks good. Left some notes clarifying the interface contract however the gist is there.


public interface PostAnalysisAware {

default void postAnalysisVerification(Failures failures) {}
Copy link
Member

@costin costin Jan 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should break interface in two, one that does validation for the current node while another interface performs tree validation.

public interface PostAnalysisVerificationAware
	void postAnalysisVerification(Failures failures);
}

and

public interface PostAnalysisPlanVerificationAware
	BiConsumer<LogicalPlan, Failures> postAnalysisPlanVerification();
}

This way classes can choose which one to pick since so far, it seems only one method is implemented per class and rarely (never?) both.

P.S. In practice the PostAnalysisVerificationAware interface is a subset of the latter:

public interface PostAnalysisVerificationAware extends PostAnalysisPlanVerificationAware
	void postAnalysisVerification(Failures failures);

	default BiConsumer<LogicalPlan, Failures> postAnalysisPlanVerification() {
         return (l,f) -> { if (getClass().isAssignable(f.getClass()) { postAnalysisVerification(f); } 
    }
}

(though this wrapping should be done in the Verifier to avoid leaking the details to the user).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I also thought of this, given the alternative to add default implementations for otherwise an opt-in interface.


default void postAnalysisVerification(Failures failures) {}

default BiConsumer<LogicalPlan, Failures> planChecker() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for consistency rename this to postAnalysisPlanVerification


import java.util.function.BiConsumer;

public interface PostAnalysisAware {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The interface needs some javadoc with examples.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added javadoc with examples.

* 2.0.
*/

package org.elasticsearch.xpack.esql.analysis;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use the org.elasticsearch.xpack.esql.capabilities package instead.

@bpintea bpintea marked this pull request as ready for review January 9, 2025 18:07
@elasticsearchmachine elasticsearchmachine added the Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) label Jan 9, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-analytical-engine (Team:Analytics)

@bpintea bpintea added the auto-backport Automatically create backport pull requests when merged label Jan 9, 2025
@bpintea bpintea changed the title ESQL: Add a PostAnalysisAware, distribute verification ESQL: Add interfaces to distribute the post-analysis verification Jan 10, 2025
Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Bogdan - this LGTM!

@costin costin merged commit ad264f7 into elastic:main Jan 10, 2025
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.x Commit could not be cherrypicked due to conflicts

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 119798

bpintea added a commit to bpintea/elasticsearch that referenced this pull request Jan 13, 2025
…astic#119798)

This adds a PostAnalysisVerificationAware interface that allows an expression,
plan or even command to perform post-analysis verifications "locally", vs.
having them centralized in the core verifier.

(cherry picked from commit ad264f7)
@bpintea bpintea deleted the enh/post_analysis_aware branch January 13, 2025 12:55
elasticsearchmachine pushed a commit that referenced this pull request Jan 13, 2025
…Aware (#119985)

Rename interface to align it with the similar post-analysis interfaces.

Related #119798.
bpintea added a commit to bpintea/elasticsearch that referenced this pull request Jan 13, 2025
…Aware (elastic#119985)

Rename interface to align it with the similar post-analysis interfaces.

Related elastic#119798.
elasticsearchmachine pushed a commit that referenced this pull request Jan 13, 2025
…19798) (#120048)

This adds a PostAnalysisVerificationAware interface that allows an expression,
plan or even command to perform post-analysis verifications "locally", vs.
having them centralized in the core verifier.

(cherry picked from commit ad264f7)
elasticsearchmachine pushed a commit that referenced this pull request Jan 13, 2025
…Aware (#119985) (#120051)

Rename interface to align it with the similar post-analysis interfaces.

Related #119798.
martijnvg pushed a commit to martijnvg/elasticsearch that referenced this pull request Jan 14, 2025
…Aware (elastic#119985)

Rename interface to align it with the similar post-analysis interfaces.

Related elastic#119798.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL auto-backport Automatically create backport pull requests when merged >enhancement Team:Analytics Meta label for analytical engine team (ESQL/Aggs/Geo) v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants