GH-45071: [Packaging][Docs] Fix NumPy v2 include directory for Emscripten, and update Pyodide-related documentation

[agriyakhetarpal]

Rationale for this change

This change would allow building PyArrow correctly with NumPy 1.X and NumPy 2.X, since we are trying to do the latter for pyodide/pyodide#4925. This PR closes gh-45071.

What changes are included in this PR?

This PR

• issues a correction for the NumPy header files when building under Emscripten
• updates Pyodide-related build instructions

Are these changes tested?

Yes, working here: https://github.com/agriyakhetarpal/pyodide-numpy-2.0-rebuilds/actions/runs/12399351376/job/34619554658#step:8:4547 via agriyakhetarpal/pyodide@b651698 that applies a subset of the changes as a patch (the CI job is failing for unrelated reasons, please ignore).

Are there any user-facing changes?

Yes, users trying to build a WASM wheel via Pyodide are now requested to use newer Pyodide and Emscripten versions, and the latest stable version of pyodide-build available.

• GitHub Issue: [Packaging][Docs] Pyodide builds have incorrect NumPy 2.0 header location and assume pyodide-build to be the same version as Pyodide #45071

[github-actions]

⚠️ GitHub issue #45071 has been automatically assigned in GitHub to PR creator.

[kou]

@github-actions crossbow submit -g python

[github-actions]

Revision: bb6ff29

Submitted crossbow builds: ursacomputing/crossbow @ actions-a9ba1a0870

Task	Status
example-python-minimal-build-fedora-conda
example-python-minimal-build-ubuntu-venv
test-conda-python-3.10
test-conda-python-3.10-cython2
test-conda-python-3.10-hdfs-2.9.2
test-conda-python-3.10-hdfs-3.2.1
test-conda-python-3.10-pandas-latest-numpy-latest
test-conda-python-3.10-substrait
test-conda-python-3.11
test-conda-python-3.11-dask-latest
test-conda-python-3.11-dask-upstream_devel
test-conda-python-3.11-hypothesis
test-conda-python-3.11-pandas-latest-numpy-1.26
test-conda-python-3.11-pandas-latest-numpy-latest
test-conda-python-3.11-pandas-nightly-numpy-nightly
test-conda-python-3.11-pandas-upstream_devel-numpy-nightly
test-conda-python-3.11-spark-master
test-conda-python-3.12
test-conda-python-3.12-cpython-debug
test-conda-python-3.13
test-conda-python-3.9
test-conda-python-3.9-pandas-1.1.3-numpy-1.19.5
test-conda-python-emscripten
test-cuda-python-ubuntu-22.04-cuda-11.7.1
test-debian-12-python-3-amd64
test-debian-12-python-3-i386
test-fedora-39-python-3
test-ubuntu-22.04-python-3
test-ubuntu-22.04-python-313-freethreading
test-ubuntu-24.04-python-3

[agriyakhetarpal]

Thanks for triggering CI! Looks like the failure is unrelated, it's just a timeout.

[raulcd]

Thanks for triggering CI! Looks like the failure is unrelated, it's just a timeout.

Yes, it seems a timeout issue but it's pretty consistent, I just triggered it a 4th time, and it has never failed on our nightly CI. The other CI failures are unrelated.

edit: Well, it was successful on the 4th attempt :)

[agriyakhetarpal]

Glad that it passed :D

I would recommend switching to using the Pyodide virtual environment pyodide venv for running the test suite, by the way – while python/scripts/run_emscripten_tests.py tests on Chrome and Firefox as well, the Node.js tests in it could be swapped out for it, and we found to be more reliable in projects such as SymPy.

[kou]

+1

I did more tries and they are stable now. (Why...?)

Could you open new issues for these improvement ideas?

[agriyakhetarpal]

I have no idea either, though FWIW, timeouts and flakes with selenium in our own test suite are pretty common.

I'll open a new issue about it, sure!

[conbench-apache-arrow]

After merging your PR, Conbench analyzed the 4 benchmarking runs that have been run so far on merge-commit 7bec4fa.

There were no benchmark performance regressions. 🎉

The full Conbench report has more details. It also includes information about 1 possible false positive for unstable benchmarks that are known to sometimes produce them.