Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Enhancement] TopN runtime filter for string support update #55122

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

[Enhancement] TopN runtime filter for string support update #55122

wants to merge 7 commits into from
+211 −150

Conversation

trueeyu
Copy link
Contributor

@trueeyu trueeyu commented Jan 16, 2025 •
edited
Loading

Why I'm doing:

What I'm doing:

TopN runtime filter for string support update

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 3.4
    • 3.3
    • 3.2
    • 3.1
    • 3.0

@trueeyu trueeyu changed the title [Enhancement] TopN runtime filter support string [Enhancement] TopN runtime filter for string support update Jan 16, 2025
@trueeyu trueeyu marked this pull request as ready for review January 16, 2025 08:40
@trueeyu trueeyu requested review from a team as code owners January 16, 2025 08:40
stdpain
stdpain previously approved these changes Jan 16, 2025
View reviewed changes
dirtysalt
dirtysalt reviewed Jan 16, 2025
View reviewed changes
be/src/exprs/runtime_filter.h
CppType min_value() const { return _min; }
CppType min_value(ObjectPool* pool) const {
if constexpr (IsSlice<CppType>) {
std::lock_guard<std::mutex> lk(_slice_mutex);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why we have slice_mutex? to protect from race condiction of pool allocation?

if that, maybe you can name it as pool_mutex?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is used to protect _slice_min/_slice_max. _min point to the std::string of _slice_min, so lock it first, copy the string, and then release it. Otherwise, when get, other threads will replace the std::string in _slice_min. ,

dirtysalt
dirtysalt reviewed Jan 16, 2025
View reviewed changes
Copy link
Contributor

@dirtysalt dirtysalt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

dirtysalt
dirtysalt previously approved these changes Jan 16, 2025
View reviewed changes
@dirtysalt dirtysalt enabled auto-merge (squash) January 16, 2025 18:36
@trueeyu
TopN runtime filter support string
156c9e5 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
fix
42345d3 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
fix
6023090 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
fix
d804226 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
fix
f406e24 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
fix
f5a7acf 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
format
e7c7220 
Signed-off-by: trueeyu <[email protected]>
@trueeyu
Copy link
Contributor Author

trueeyu commented Jan 17, 2025

https://github.com/Mergifyio rebase main

@mergify Mergify
Copy link
Contributor

mergify bot commented Jan 17, 2025

rebase main

☑️ Nothing to do

  • -conflict [📌 rebase requirement]
  • -closed [📌 rebase requirement]
  • queue-position = -1 [📌 rebase requirement]
  • any of:
    • #commits > 1 [📌 rebase requirement]
    • #commits-behind > 0 [📌 rebase requirement]
    • -linear-history [📌 rebase requirement]

@trueeyu trueeyu dismissed stale reviews from dirtysalt and stdpain via e7c7220 January 17, 2025 02:36
@github-actions GitHub Actions
Copy link

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[FE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

@github-actions GitHub Actions
Copy link

[BE Incremental Coverage Report]

✅ pass : 53 / 59 (89.83%)

file detail

path covered_line new_line coverage not_covered_line_detail
🔵 be/src/exec/olap_scan_prepare.cpp 5 6 83.33% [722]
🔵 be/src/exprs/runtime_filter.h 29 34 85.29% [405, 406, 442, 443, 444]
🔵 be/src/storage/runtime_range_pruner.hpp 17 17 100.00% []
🔵 be/src/exprs/min_max_predicate.h 2 2 100.00% []

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dirtysalt dirtysalt dirtysalt approved these changes

@wyb wyb wyb approved these changes

@stdpain stdpain stdpain left review comments

At least 2 approving reviews are required to merge this pull request.

Assignees

@trueeyu trueeyu

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@trueeyu @wyb @dirtysalt @stdpain