-
Notifications
You must be signed in to change notification settings - Fork 10
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
use Parquet.jl named tuples iterator: RecordCursor #25
base: master
Are you sure you want to change the base?
Conversation
This [Parquet.jl update](JuliaIO/Parquet.jl#71) will add a named tuple iterator `RecordReader`. We will be able to use that here directly, instead of wrapping over the older `RecCursor`. The new `map_logical_types` option to `ParFile` automatically converts byte arrays to strings, so we do not need to handle that here now.
I think this broke live. julia> using Queryverse Package Planenvironment location: /home/blair/.julia/conda/3 added / updated specs: The following packages will be downloaded:
The following NEW packages will be INSTALLED: xlrd pkgs/main/linux-64::xlrd-1.2.0-py37_0 Downloading and Extracting Packages |
Rec cursor isn't defined any more, but, it is being referred to. |
I think merging this would fix that issue. Parquet.jl has moved to I feel there should be an upperbound to the compat version range here so that we don't run into this issue again. I shall push a commit with that in a bit. |
@davidanthoff does this look good to be merged? |
I'd say merge this with master to fix the conflicts in Project.toml which has proper upper bounding. |
merging this will fix #26 and all the file read issues |
This Parquet.jl update will add a named tuple iterator
RecordReader
. We will be able to use that here directly, instead of wrapping over the olderRecCursor
.The new
map_logical_types
option toParFile
automatically converts byte arrays to strings, so we do not need to handle that here now.This can be merged after JuliaIO/Parquet.jl#71 is released in Parquet.jl