WIP: [Release] Verify release-14.0.0-rc2

[kou]

PR to verify Release Candidate

[kou]

@github-actions crossbow submit --group verify-rc-source --param release=14.0.0 --param rc=2

[github-actions]

Revision: 2dcee3f

Submitted crossbow builds: ursacomputing/crossbow @ actions-59980692f0

Task	Status
verify-rc-source-cpp-linux-almalinux-8-amd64
verify-rc-source-cpp-linux-conda-latest-amd64
verify-rc-source-cpp-linux-ubuntu-20.04-amd64
verify-rc-source-cpp-linux-ubuntu-22.04-amd64
verify-rc-source-cpp-macos-amd64
verify-rc-source-cpp-macos-arm64
verify-rc-source-cpp-macos-conda-amd64
verify-rc-source-csharp-linux-almalinux-8-amd64
verify-rc-source-csharp-linux-conda-latest-amd64
verify-rc-source-csharp-linux-ubuntu-20.04-amd64
verify-rc-source-csharp-linux-ubuntu-22.04-amd64
verify-rc-source-csharp-macos-amd64
verify-rc-source-csharp-macos-arm64
verify-rc-source-go-linux-almalinux-8-amd64
verify-rc-source-go-linux-conda-latest-amd64
verify-rc-source-go-linux-ubuntu-20.04-amd64
verify-rc-source-go-linux-ubuntu-22.04-amd64
verify-rc-source-go-macos-amd64
verify-rc-source-go-macos-arm64
verify-rc-source-integration-linux-almalinux-8-amd64
verify-rc-source-integration-linux-conda-latest-amd64
verify-rc-source-integration-linux-ubuntu-20.04-amd64
verify-rc-source-integration-linux-ubuntu-22.04-amd64
verify-rc-source-integration-macos-amd64
verify-rc-source-integration-macos-arm64
verify-rc-source-integration-macos-conda-amd64
verify-rc-source-java-linux-almalinux-8-amd64
verify-rc-source-java-linux-conda-latest-amd64
verify-rc-source-java-linux-ubuntu-20.04-amd64
verify-rc-source-java-linux-ubuntu-22.04-amd64
verify-rc-source-java-macos-amd64
verify-rc-source-js-linux-almalinux-8-amd64
verify-rc-source-js-linux-conda-latest-amd64
verify-rc-source-js-linux-ubuntu-20.04-amd64
verify-rc-source-js-linux-ubuntu-22.04-amd64
verify-rc-source-js-macos-amd64
verify-rc-source-js-macos-arm64
verify-rc-source-python-linux-almalinux-8-amd64
verify-rc-source-python-linux-conda-latest-amd64
verify-rc-source-python-linux-ubuntu-20.04-amd64
verify-rc-source-python-linux-ubuntu-22.04-amd64
verify-rc-source-python-macos-amd64
verify-rc-source-python-macos-arm64
verify-rc-source-python-macos-conda-amd64
verify-rc-source-ruby-linux-almalinux-8-amd64
verify-rc-source-ruby-linux-conda-latest-amd64
verify-rc-source-ruby-linux-ubuntu-20.04-amd64
verify-rc-source-ruby-linux-ubuntu-22.04-amd64
verify-rc-source-ruby-macos-amd64
verify-rc-source-ruby-macos-arm64
verify-rc-source-windows

[raulcd]

Revision: 2dcee3f

Submitted crossbow builds: ursacomputing/crossbow @ release-14.0.0-rc2-0

Task	Status
almalinux-8-amd64
almalinux-8-arm64
almalinux-9-amd64
almalinux-9-arm64
amazon-linux-2023-amd64
amazon-linux-2023-arm64
centos-7-amd64
centos-8-stream-amd64
centos-8-stream-arm64
centos-9-stream-amd64
centos-9-stream-arm64
conan-maximum
conan-minimum
debian-bookworm-amd64
debian-bookworm-arm64
debian-bullseye-amd64
debian-bullseye-arm64
debian-trixie-amd64
debian-trixie-arm64
java-jars
nuget
python-sdist
r-binary-packages
test-ubuntu-22.04-docs
ubuntu-focal-amd64
ubuntu-focal-arm64
ubuntu-jammy-amd64
ubuntu-jammy-arm64
ubuntu-lunar-amd64
ubuntu-lunar-arm64
wheel-clean
wheel-macos-big-sur-cp310-arm64
wheel-macos-big-sur-cp311-arm64
wheel-macos-big-sur-cp312-arm64
wheel-macos-big-sur-cp38-arm64
wheel-macos-big-sur-cp39-arm64
wheel-macos-mojave-cp310-amd64
wheel-macos-mojave-cp311-amd64
wheel-macos-mojave-cp312-amd64
wheel-macos-mojave-cp38-amd64
wheel-macos-mojave-cp39-amd64
wheel-manylinux-2-28-cp310-amd64
wheel-manylinux-2-28-cp310-arm64
wheel-manylinux-2-28-cp311-amd64
wheel-manylinux-2-28-cp311-arm64
wheel-manylinux-2-28-cp312-amd64
wheel-manylinux-2-28-cp312-arm64
wheel-manylinux-2-28-cp38-amd64
wheel-manylinux-2-28-cp38-arm64
wheel-manylinux-2-28-cp39-amd64
wheel-manylinux-2-28-cp39-arm64
wheel-manylinux-2014-cp310-amd64
wheel-manylinux-2014-cp310-arm64
wheel-manylinux-2014-cp311-amd64
wheel-manylinux-2014-cp311-arm64
wheel-manylinux-2014-cp312-amd64
wheel-manylinux-2014-cp312-arm64
wheel-manylinux-2014-cp38-amd64
wheel-manylinux-2014-cp38-arm64
wheel-manylinux-2014-cp39-amd64
wheel-manylinux-2014-cp39-arm64
wheel-windows-cp310-amd64
wheel-windows-cp311-amd64
wheel-windows-cp312-amd64
wheel-windows-cp38-amd64
wheel-windows-cp39-amd64

[raulcd]

@ursabot please benchmark

[ursabot]

Benchmark runs are scheduled for commit 2dcee3f. Watch https://buildkite.com/apache-arrow and https://conbench.ursa.dev for updates. A comment will be posted here when the runs are complete.

[conbench-apache-arrow]

Thanks for your patience. Conbench analyzed the 6 benchmarking runs that have been run so far on PR commit 2dcee3f.

There were 2 benchmark results indicating a performance regression:

• Pull Request Run on ursa-thinkcentre-m75q at 2023-10-19 15:20:36Z
  • MoveEmptyVector (C++) with params=<STATIC_VECTOR(std::string)>, source=cpp-micro, suite=arrow-small-vector-benchmark
  • BitBlockCounterSumWithOffset (C++) with params=32768, source=cpp-micro, suite=arrow-bit-block-counter-benchmark

The full Conbench report has more details.

[raulcd]

@github-actions crossbow submit --group verify-rc-binaries --group verify-rc-wheels --param release=14.0.0 --param rc=2

[github-actions]

Revision: 2dcee3f

Submitted crossbow builds: ursacomputing/crossbow @ actions-ab9d709f3b

Task	Status
verify-rc-binaries-apt-linux-amd64
verify-rc-binaries-binary-linux-amd64
verify-rc-binaries-jars-linux-almalinux-8-amd64
verify-rc-binaries-jars-linux-conda-latest-amd64
verify-rc-binaries-jars-linux-ubuntu-20.04-amd64
verify-rc-binaries-jars-linux-ubuntu-22.04-amd64
verify-rc-binaries-wheels-linux-almalinux-8-amd64
verify-rc-binaries-wheels-linux-conda-latest-amd64
verify-rc-binaries-wheels-linux-ubuntu-20.04-amd64
verify-rc-binaries-wheels-linux-ubuntu-22.04-amd64
verify-rc-binaries-wheels-macos-11-amd64
verify-rc-binaries-wheels-macos-11-arm64
verify-rc-binaries-wheels-macos-12-amd64
verify-rc-binaries-wheels-windows
verify-rc-binaries-yum-linux-amd64

[raulcd]

@kou @kszucs any idea why it might be taking more than 6 hours to install system dependencies for verify-rc-binaries-wheels-macos-11-amd64 ? On the previous release it took 6 minutes: https://github.com/ursacomputing/crossbow/actions/runs/5899732992/job/16002716613
I am trying to understand if there is an issue with the new runner image:

Image: macos-11
  Version: 20230921.4
  Included Software: https://github.com/actions/runner-images/blob/macOS-11/20230921.4/images/macos/macos-11-Readme.md
  Image Release: [https://github.com/actions/runner-images/releases/tag/macOS-11%2F](https://github.com/actions/runner-images/releases/tag/macOS-11%2F20230921.4)

vs the old:

   Image: macos-11
  Version: 20230812.3
  Included Software: https://github.com/actions/runner-images/blob/macOS-11/20230812.3/images/macos/macos-11-Readme.md
  Image Release: https://github.com/actions/runner-images/releases/tag/macOS-11%2F20230812.3

All jobs are successful except that one. I am thinking on sending the vote email nevertheless as both:

• verify-rc-binaries-wheels-macos-11-arm64
• verify-rc-binaries-wheels-macos-12-amd64

where successful with the wheels and the current failure is timing out installing system dependencies, nothing to do verifying the wheels.

[raulcd]

I am surprised as installing system dependencies takes a reasonable amount of time both on macos12-amd64 and macos11-arm64 (10-15 minutes)

[kszucs]

Do the nightly builds pass? (Forgot that we don't have wheel RC verification nightlies).

Nonetheless, it looks like a build issue rather than an issue with the release itself. The wheel builds should execute the tests, so I think it is totally fine to not block the vote on this build error. We should be able to verify them locally until the verification builds are fixed.

[kszucs]

Looking at the rest of the builds there are test errors in the go source verification tasks. cc @zeroshade

[raulcd]

Thanks @kszucs the go verification issue has been fixed. Is an issue in the verification script!

[kou]

I agree with kszucs. It's not a blocker.
I think that Homebrew waits for a user input.
BTW, we should set timeout for this case...