Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

GH-45071: [Packaging][Docs] Fix NumPy v2 include directory for Emscripten, and update Pyodide-related documentation #45072

Merged
merged 5 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ci/docker/conda-python-emscripten.dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -27,14 +27,14 @@ ARG required_python_min="(3,12)"
# fail if python version < 3.12
RUN echo "check PYTHON>=${required_python_min}" && python -c "import sys;sys.exit(0 if sys.version_info>=${required_python_min} else 1)"

# install selenium and pyodide-build and recent python
# install selenium and recent pyodide-build and recent python

# needs to be a login shell so ~/.profile is read
SHELL ["/bin/bash", "--login", "-c", "-o", "pipefail"]

RUN python -m pip install --no-cache-dir selenium==${selenium_version} && \
python -m pip install --no-cache-dir --upgrade pyodide-build==${pyodide_version}
python -m pip install --no-cache-dir --upgrade pyodide-build>=${pyodide_version}

# install pyodide dist directory to /pyodide
RUN pyodide_dist_url="https://github.com/pyodide/pyodide/releases/download/${pyodide_version}/pyodide-${pyodide_version}.tar.bz2" && \
wget -q "${pyodide_dist_url}" -O- | tar -xj -C /
Expand Down
16 changes: 8 additions & 8 deletions docs/source/developers/cpp/emscripten.rst
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,9 @@ activate it using the commands below (see https://emscripten.org/docs/getting_st
git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
# replace <version> with the desired EMSDK version.
# e.g. for Pyodide 0.24, you need EMSDK version 3.1.45
# e.g. for Pyodide 0.26, you need EMSDK version 3.1.58
# the versions can be found in the Makefile.envs file in the Pyodide repo:
# https://github.com/pyodide/pyodide/blob/10b484cfe427e076c929a55dc35cfff01ea8d3bc/Makefile.envs
./emsdk install <version>
./emsdk activate <version>
source ./emsdk_env.sh
Expand All @@ -46,8 +48,8 @@ versions of emsdk tools.
.. code:: shell

# install Pyodide build tools.
# e.g. for version 0.24 of Pyodide:
pip install pyodide-build==0.24
# e.g., for version 0.26 of Pyodide, pyodide-build 0.26 and later work
pip install "pyodide-build>=0.26"

Then build with the ``ninja-release-emscripten`` CMake preset,
like below:
Expand All @@ -69,8 +71,7 @@ go to ``arrow/python`` and run
pyodide build

It should make a wheel targeting the currently enabled version of
Pyodide (i.e. the version corresponding to the currently installed
``pyodide-build``) in the ``dist`` subdirectory.
Pyodide in the ``dist`` subdirectory.


Manual Build
Expand All @@ -85,9 +86,8 @@ you will need to override. In particular you will need:

#. ``CMAKE_TOOLCHAIN_FILE`` set by using ``emcmake cmake`` instead of just ``cmake``.

#. You will quite likely need to set ``ARROW_ENABLE_THREADING`` to ``OFF``
for builds targeting single threaded Emscripten environments such as
Pyodide.
#. You will need to set ``ARROW_ENABLE_THREADING`` to ``OFF`` for builds
targeting single-threaded Emscripten environments such as Pyodide.

#. ``ARROW_FLIGHT`` and anything else that uses network probably won't
work.
Expand Down
11 changes: 10 additions & 1 deletion python/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -166,8 +166,17 @@ if($ENV{PYODIDE})
# modules (at least under Pyodide it does).
set(Python3_INCLUDE_DIR $ENV{PYTHONINCLUDE})
set(Python3_LIBRARY $ENV{CPYTHONLIB})
set(Python3_NumPy_INCLUDE_DIR $ENV{NUMPY_LIB}/core/include)
set(Python3_EXECUTABLE)
execute_process(COMMAND ${Python3_EXECUTABLE} -c
"import numpy; print(numpy.__version__)"
OUTPUT_VARIABLE PYODIDE_NUMPY_VERSION
OUTPUT_STRIP_TRAILING_WHITESPACE)
string(REGEX MATCH "^([0-9]+)" PYODIDE_NUMPY_MAJOR_VERSION ${PYODIDE_NUMPY_VERSION})
if(PYODIDE_NUMPY_MAJOR_VERSION GREATER_EQUAL 2)
set(Python3_NumPy_INCLUDE_DIR $ENV{NUMPY_LIB}/_core/include)
else()
set(Python3_NumPy_INCLUDE_DIR $ENV{NUMPY_LIB}/core/include)
endif()
set(ENV{_PYTHON_SYSCONFIGDATA_NAME} $ENV{SYSCONFIG_NAME})
# we set the c and cxx compiler manually to bypass pywasmcross
# which is pyodide's way of messing with C++ build parameters.
Expand Down
Loading