Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEA/Discussion] Make left joins have no order guarantees by default #20762

Open
Matt711 opened this issue Jan 17, 2025 · 2 comments
Open

[FEA/Discussion] Make left joins have no order guarantees by default #20762

Matt711 opened this issue Jan 17, 2025 · 2 comments
Labels
enhancement New feature or an improvement of an existing feature

Comments

@Matt711
Copy link

Matt711 commented Jan 17, 2025

Description

Since the 1.17 release, polars has supported different types of order preserving joins via the maintain_order parameter.

This option was also exposed in the python IR. We've started adding support for maintain_order in joins in the GPU executor, but we've had to skip some of the polars tests because I think left joins are order preserving by default. Is that true? In one of the tests in test_join.py, there's a comment about it. Is that comment accurate?

Here are the tests we skip:

    "tests/unit/operations/test_join.py::test_update"
    "tests/unit/operations/test_join.py::test_join_preserve_order_left",
    "tests/unit/streaming/test_streaming.py::test_streaming_generic_left_and_inner_join_from_disk",
    "tests/unit/streaming/test_streaming_join.py::test_join_null_matches[False]",
    "tests/unit/streaming/test_streaming_join.py::test_join_null_matches_multiple_keys[False]",
    "tests/unit/test_cse.py::test_cse_rename_cross_join_5405"

cc. @wence-

@Matt711 Matt711 added the enhancement New feature or an improvement of an existing feature label Jan 17, 2025
@Matt711 Matt711 changed the title [FEA/Discussion] Make left joins have no order guaruntees [FEA/Discussion] Make left joins have no order guaruntees by default Jan 17, 2025
@MarcoGorelli
Copy link
Collaborator

Hi @Matt711 - I think it's already not guaranteed. It used to be, but that was changed in #20048

@orlp
Copy link
Collaborator

orlp commented Jan 19, 2025

Feel free to rewrite any test which currently assumes joins are order-preserving to explicitly specify the order they expect preserved with the new maintain_order argument.

@nameexhaustion nameexhaustion changed the title [FEA/Discussion] Make left joins have no order guaruntees by default [FEA/Discussion] Make left joins have no order guarantees by default Jan 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or an improvement of an existing feature
Projects
None yet
Development

No branches or pull requests

3 participants