GH-43695: [C++][Parquet] flatbuffers metadata experiments

[alkis]

This is an annotated attempt to use flatbuffers as metadata for parquet. The goals are:

1. flatbuffers "parse" extremely fast compared to thrift which
   • cuts down on critical path latency of processing parquet files
   • is so fast, O(n) effects of "parsing" metadata vs scanning 1 column are eliminated
2. flatbuffers are typically bulkier than thrift, in this PR there are a multitude of optimizations to shrink the size of flatbuffer metadata
3. keep the flatbuffer object model similar to that of thrift to facilitate easier migration to new metadata format

To run experiments:

mkdir arrow/src/o
cd arrow/src
cmake o --preset ninja-benchmarks
ninja -Co && o/relwithdebinfo/parquet-metadata3-benchmark path-to-footers/*

• GitHub Issue: [C++][Parquet] Proof-of-concept: Trying to using FlatBuffer as Parquet Footer #43695

[github-actions]

Thanks for opening a pull request!

If this is not a minor PR. Could you open an issue for this pull request on GitHub? https://github.com/apache/arrow/issues/new/choose

Opening GitHub issues ahead of time contributes to the Openness of the Apache Arrow project.

Then could you also rename the pull request title in the following format?

GH-${GITHUB_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

or

MINOR: [${COMPONENT}] ${SUMMARY}

In the case of PARQUET issues on JIRA the title also supports:

PARQUET-${JIRA_ISSUE_ID}: [${COMPONENT}] ${SUMMARY}

See also:

• Other pull requests
• Contribution Guidelines - Contributing Overview

[corwinjoy]

Thanks for posting this!
A few basic initial comments / questions:
Running the build, I think you want the following for instructions:

cd arrow/cpp
mkdir o
cd o
cmake .. --preset ninja-benchmarks
cd ..
ninja -Co
o/relwithdebinfo/parquet-metadata3-benchmark path-to-footers/*

Here I split the build and run. For now, it will not run until you generate footers per (https://github.com/apache/parquet-benchmark/pull/1/files).

[corwinjoy]

As part of running this code, I put in a PR with synthetic footers to the parquet-benchmark repository that can be used to get started (when setting your path-to-footers/directory).
apache/parquet-benchmark#2

[github-actions]

⚠️ GitHub issue #43695 has been automatically assigned in GitHub to PR creator.