[EXP][C++] Deduplicate schemas when scanning Dataset

[pitrou]

Rationale for this change

What changes are included in this PR?

Are these changes tested?

Are there any user-facing changes?

[pitrou]

@ursabot please benchmark

[ursabot]

Benchmark runs are scheduled for commit f681035. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[pitrou]

@ursabot please benchmark

[ursabot]

Benchmark runs are scheduled for commit a0503f3. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[pitrou]

@icexelloss This is a quick experiment that you might want to try out on a real-world use case. I don't seem to get any tangible benefits on a synthetic dataset, though it might be due to memory fragmentation.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 0 benchmarking runs that have been run so far on PR commit f681035.

None of the specified runs were found on the Conbench server.

The full Conbench report has more details.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit a0503f3.

There were 775 benchmark results indicating a performance regression:

• Pull Request Run on test-mac-arm at 2025-01-23 18:46:18Z
  • TakeChunkedChunkedStringFewMonotonicIndices (C++) with params=4194304/1, source=cpp-micro, suite=arrow-compute-vector-selection-benchmark
  • ListSliceStringListViewWithStop (C++) with params=65536/0, source=cpp-micro, suite=arrow-compute-scalar-list-benchmark
• and 773 more (see the report linked below)

The full Conbench report has more details.

[pitrou]

Ok, there are so many unrelated "regressions" in the report above that I'm going to launch another benchmarking run, as it's likely that external factors have influenced that run.

[pitrou]

@ursabot please benchmark

[ursabot]

Commit a0503f3 already has scheduled benchmark runs.

[pitrou]

@ursabot please benchmark

[ursabot]

Benchmark runs are scheduled for commit 335a46b. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 4 benchmarking runs that have been run so far on PR commit 335a46b.

There were 23 benchmark results indicating a performance regression:

• Pull Request Run on test-mac-arm at 2025-01-24 15:13:21Z
  • BenchmarkTemporal (C++) with params=<Subsecond, non_zoned>/4194304/0, source=cpp-micro, suite=arrow-compute-scalar-temporal-benchmark
  • FilterOverhead (C++) with params=selectivity_benchmark/batch_size:100000/null_prob:100/bool_true_prob:50/real_time, source=cpp-micro, suite=arrow-acero-filter-benchmark
• and 21 more (see the report linked below)

The full Conbench report has more details.