Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add new field updatedRows to QueryStatistics.java where it's availabl… #24715

Open
wants to merge 48 commits into
base: master
Choose a base branch
from

Conversation

bhzaeri
Copy link

@bhzaeri bhzaeri commented Jan 15, 2025

…e in EventListener callbacks. It comes from TableMutationOperator.java where update/delete queries are issued.

Description

The issue that this pull request tries to fix is that after running UPDATE/DELETE queries, the outputRows in QueryStatistics is always 1. But we need the actual value of updated rows. We figured out that the number of updated rows is returned in TableMutationOperator.java method getOutput(). The updated rows number is passed to operatorContext instance and from there, all the way down to the QueryStatistics.java which make the number available in the event listeners.

Additional context and related issues

In our ransomware defender platform, we need to monitor the behavior of users who have access to run queries on the DBs connected via Trino. So, we need to be notified of users' actions and the exact results of their actions. Trino returns the correct values for SELECT and INSERT queries. We need the same for UPDATE/DELTE as well.

The only issue is that the current pull request doesn't include the proper unit tests for the changes. Could you please help us with the proper way to write the unit tests?

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(X) Release notes are required, with the following suggested text:

## Section
* Fix some things. ({issue}`24596`)

#24596

Copy link

cla-bot bot commented Jan 15, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

1 similar comment
Copy link

cla-bot bot commented Jan 15, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@bhzaeri
Copy link
Author

bhzaeri commented Jan 15, 2025

Hi, Could you please take a look?
The issue in #24687 is fixed and the updatedCount is not passed to Page instead it's passed from TableMutationOperator to operatorContext.
@chenjian2664 @wendigo

Copy link

cla-bot bot commented Jan 16, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@bhzaeri bhzaeri force-pushed the event-listener-updated-rows-number-2 branch from 04340f6 to 22385d9 Compare January 16, 2025 00:55
Copy link

cla-bot bot commented Jan 16, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@@ -121,6 +121,8 @@ public Page getOutput()

OptionalLong rowsUpdatedCount = operation.execute();

operatorContext.recordUpdatedPositions(rowsUpdatedCount.orElse(0L));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you may also need to take care of the rowCount in the MergeWriterOperator

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How can I run into the MergeWriterOperator? Should I run a MERGE query?

@bhzaeri bhzaeri force-pushed the event-listener-updated-rows-number-2 branch from 22385d9 to 1ee7ced Compare January 16, 2025 15:04
Copy link

cla-bot bot commented Jan 16, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@wendigo
Copy link
Contributor

wendigo commented Jan 21, 2025

@cla-bot check

Copy link

cla-bot bot commented Jan 21, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Copy link

cla-bot bot commented Jan 21, 2025

The cla-bot has been summoned, and re-checked this pull request!

…e in EventListener callbacks. It comes from TableMutationOperator.java where update/delete queries are issued.
@bhzaeri bhzaeri force-pushed the event-listener-updated-rows-number-2 branch from 1ee7ced to 0f6ec97 Compare January 21, 2025 15:05
Copy link

cla-bot bot commented Jan 21, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Copy link

cla-bot bot commented Jan 24, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

@github-actions github-actions bot added docs iceberg Iceberg connector bigquery BigQuery connector labels Jan 24, 2025
Copy link

cla-bot bot commented Jan 24, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

…e in EventListener callbacks. It comes from TableMutationOperator.java where update/delete queries are issued.
…e in EventListener callbacks. It comes from TableMutationOperator.java where update/delete queries are issued.
…e in EventListener callbacks. It comes from TableMutationOperator.java where update/delete queries are issued.
Copy link

cla-bot bot commented Jan 24, 2025

Thank you for your pull request and welcome to the Trino community. We require contributors to sign our Contributor License Agreement, and we don't seem to have you on file. Continue to work with us on the review and improvements in this PR, and submit the signed CLA to [email protected]. Photos, scans, or digitally-signed PDF files are all suitable. Processing may take a few days. The CLA needs to be on file before we merge your changes. For more information, see https://github.com/trinodb/cla

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bigquery BigQuery connector docs iceberg Iceberg connector
Development

Successfully merging this pull request may close these issues.

10 participants