Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert LexOrdering type to struct. #13146

Merged
merged 19 commits into from
Nov 1, 2024

Conversation

ngli-me
Copy link
Contributor

@ngli-me ngli-me commented Oct 28, 2024

Which issue does this PR close?

Closes #12591.

Rationale for this change

What changes are included in this PR?

Adds LexOrdering as a struct, instead of a type.
Replaces previous occurrences of LexOrdering of the type with struct where appropriate.
Replaces &[PhysicalSortExpr] with the type LexOrderingRef, where appropriate.

Aligns printing for LexOrdering, since some instances of a PhysicalSortExpr were being printed with ,, and others with , . Changed printing over to the latter, since this seems to align more with other instances of printing multiple items.

Are these changes tested?

Yes, adjusted tests where necessary.

Are there any user-facing changes?

The Vec needs to be wrapped to initialize the struct instance, and some of the usage needs to be adjusted to get the field from the struct.

@github-actions github-actions bot added physical-expr Physical Expressions optimizer Optimizer rules core Core DataFusion crate proto Related to proto crate functions labels Oct 28, 2024
@ngli-me ngli-me force-pushed the feature/lex-ordering-request-struct branch from 7ba63c3 to bf53894 Compare October 29, 2024 01:08
@github-actions github-actions bot added the sqllogictest SQL Logic Tests (.slt) label Oct 29, 2024
@ngli-me ngli-me changed the title Conversion types for LexOrdering and LexOrderingRef to structs. Convert LexOrdering and LexOrderingRef types to structs. Oct 30, 2024
@ngli-me ngli-me changed the title Convert LexOrdering and LexOrderingRef types to structs. Convert LexOrdering type to struct. Oct 30, 2024
@ngli-me ngli-me marked this pull request as ready for review October 30, 2024 22:03
@ngli-me
Copy link
Contributor Author

ngli-me commented Oct 30, 2024

I initially also converted LexOrderingRef over to a struct as well in commit 5c78e93, but it looks like the struct wasn't playing well with some of the lifetimes, so I undid those changes.

Copy link
Contributor

@berkaysynnada berkaysynnada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thank you @ngli-me. Just have a few minor suggestions

datafusion/core/benches/sort.rs Outdated Show resolved Hide resolved
datafusion/core/src/datasource/physical_plan/mod.rs Outdated Show resolved Hide resolved
datafusion/core/src/physical_optimizer/enforce_sorting.rs Outdated Show resolved Hide resolved
datafusion/physical-expr/src/equivalence/properties.rs Outdated Show resolved Hide resolved
datafusion/physical-expr/src/equivalence/properties.rs Outdated Show resolved Hide resolved
datafusion/physical-expr/src/equivalence/properties.rs Outdated Show resolved Hide resolved
datafusion/physical-plan/src/joins/symmetric_hash_join.rs Outdated Show resolved Hide resolved
@ngli-me ngli-me force-pushed the feature/lex-ordering-request-struct branch from 335339a to e99e386 Compare October 31, 2024 16:22
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is amazing -- thank you @ngli-me and @berkaysynnada

I think we can make it even better and remove LexOrderingRef in favor of &LexOrdering but we can do so as a follow on PR.

I am exploring how to do this

@alamb alamb added the api change Changes the API exposed to users of the crate label Oct 31, 2024
@alamb
Copy link
Contributor

alamb commented Nov 1, 2024

Thank you again @ngli-me and @berkaysynnada

I filed some follow on work:

I also double checked that when I merged this branch up to main it still passed all its tests

@alamb alamb merged commit 752561a into apache:main Nov 1, 2024
24 checks passed
@ngli-me ngli-me deleted the feature/lex-ordering-request-struct branch November 3, 2024 17:44
@alamb alamb mentioned this pull request Nov 5, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api change Changes the API exposed to users of the crate core Core DataFusion crate functions optimizer Optimizer rules physical-expr Physical Expressions proto Related to proto crate sqllogictest SQL Logic Tests (.slt)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Making LexOrderingRef an actual struct
3 participants