-
Notifications
You must be signed in to change notification settings - Fork 18
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disallow out of range numerics type on table creation #49
base: main
Are you sure you want to change the base?
Disallow out of range numerics type on table creation #49
Conversation
d49ad88
to
003b9f6
Compare
003b9f6
to
ae72d4f
Compare
|
||
// If column is of type "numeric", check whether it's acceptable for mooncake; | ||
// If not, exception is thrown via `elog`. | ||
void ValidateColumnNumericType(Form_pg_attribute &attribute) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems good to also have the same lockdown for pg_duckdb's temp DuckDB table?
So it might be better to make this change to pg_duckdb instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If I don't read it wrong, this is how pg_duckdb handles temp table creation:
https://github.com/duckdb/pg_duckdb/blob/bb82c937936b97824bb766893745e1456eb1aa08/src/pgduckdb_ddl.cpp#L199-L364
Considering the fact that pg_duckdb doesn't support complete table creation functionality, I think it would be nice if:
- pg_duckdb locks down temp table creation on duckdb-incompatible types
- pg_mooncake locks down table creation on duckdb-incompatible types
It seems good to also have the same lockdown for pg_duckdb's temp DuckDB table?
To your question, yes.
So it might be better to make this change to pg_duckdb instead?
But I'm not sure how we could place the whole lock down logic into pg_duckdb?
Since it only handles part of the table creation logic as of now.
I mentioned it briefly in the PR description:
Method-2: propagate the logic to table creation logic into pg_duckdb
It doesn't work because pg_duckdb doesn't support general-purpose table creation as of now;
Moving the logic into pg type -> duck type conversion is also not proper.
Let me know if I mis-understand pg_duckdb's capability.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the creation of columnstore tables and DuckDB temporary tables follow different code paths
My suggestion was only to move the helper functions numeric_typmod_precision
, numeric_typmod_scale
, and ValidateColumnNumericType
in this file to pg_duckdb
I noticed you just opened duckdb/pg_duckdb#471, which already addresses this
We just need to wait for that diff to land and re-pull from pg_duckdb
324f0e1
to
a3110b8
Compare
9ff12f0
to
677e521
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We just need to wait for duckdb/pg_duckdb#471 to land and re-pull from pg_duckdb
|
||
// If column is of type "numeric", check whether it's acceptable for mooncake; | ||
// If not, exception is thrown via `elog`. | ||
void ValidateColumnNumericType(Form_pg_attribute &attribute) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, the creation of columnstore tables and DuckDB temporary tables follow different code paths
My suggestion was only to move the helper functions numeric_typmod_precision
, numeric_typmod_scale
, and ValidateColumnNumericType
in this file to pg_duckdb
I noticed you just opened duckdb/pg_duckdb#471, which already addresses this
We just need to wait for that diff to land and re-pull from pg_duckdb
Signed-off-by: hjiang <[email protected]>
64edfb9
to
3dc77c4
Compare
64a3e22
to
6a655ea
Compare
Followup PR for issue #47
This PR disallows table creation when out-of-range (for duckdb) numerics types created.
The implementation here copies the type checking logic into
columnstore_handle.cpp
.Pro: Having the check logic outside of pg_duckdb reduces the overhead for upgrade
Con: Need to maintain two util functions to get scale and width, every time we upgrade pg_duckdb.
Alternative considered:
Method-1: add extra type conversion code in pg_duckdb.
Example code
Pro: least possible code change
Con: every time we upgrade pg_duckdb, the extra checking logic has to be cherry-picked also
Method-2: propagate the logic to table creation logic into pg_duckdb
It doesn't work because pg_duckdb doesn't support general-purpose table creation as of now;
Moving the logic into pg type -> duck type conversion is also not proper.
How I tested: