exodide: build_ext for Pyodide

1. Overview

Warning This project is still under development.

Pyodide is a WebAssembly variant of CPython. By using it, we can run Python code inside web browser.

Although we can run most of pure-Python packages on Pyodide, however, available C/C++-extension packages are limited to builtin packages.

The motivation of this project (exodide) is providing C/C++-extension builder for Pyodide, and enables users to run your own custom C/C++-extension packages on Pyodide.

2. Usage

2.1 Requirement Summary

• 2.3: Build
  • Emscripten v3.1.13
  • wheel
  • C/C++ package source working on CPython 3.10.2
• 2.4: Run
  • Pyodide v0.21.0a2
• 2.5: Inspect
  • NumPy

2.2 Install exodide

You can install exodide from PyPI

• pip install exodide
  • Only exodide
• pip install exodide[build]
  • With wheel
• pip install exodide[inspect]
  • With NumPy
• pip install exodide[all]
  • With NumPy and wheel

Since this repository doesn't contains patched headers, you cannot install directly from GitHub like pip install git+https://github.com/ymd-h/exodide. Please read 4. Build exodide.

2.3 Build with exodide

The followings are reuired;

• Emscripten v3.1.13
  • Set up emcc and em++ as the official document
• C/C++ extension package source working on Python 3.10
• wheel (pip install exodide[build] install it, too.)

2.3.1 Method 1: Main Usage

from setuptools import setup
from exodide import build

# omit

setup(
    # omit
    cmdclass=build.cmdclass(), # {'build': build.build, 'build_ext': build.build_ext}
)

then CC=emcc CXX=em++ python setup.py bdist_wheel.

2.3.2 Method 2: Fine tuning for Power User

If your package has special build flow, you might need to call exodide internal API.

All internal buil API are implemented at exodide.build module.

• system_include() -> str: System include directory of host Python
• exodide_include() -> List[str]: Include directories inside exodide package.
• adjust_include(include: List[str]) -> List[str]: Adjust include directories
  • Internally system_include() and exodide_include() are used
• exodide_links() -> List[str]: Linker arguments
• exodide_unsupported_links() -> List[str]: Unsupported linker arguments
• exodide_platform_tag() -> str: Platform name tag for wheel

2.3.3 Method 3: Quick Usage

Under Consideration

2.3.4 Notes

Pyodide doesn't provide all the functionalities of CPython, so that you might need to modify your package. You can detect Emscripten compiler by __EMSCRIPTEN__ macro (ref).

#ifdef __EMSCRIPTEN__
// Code for Pyodide
#else
// Code for Others
#endif

2.4 Install extension package to Pyodide

Warning exodide doesn't check any wheel compatibility, so that user must take care.

Pyodide v0.21a2 is required.

const pyodide = await loadPyodide();

await pyodide.runPythonAsync(`
import micropip
micropip.install("exodide")

from exodide.install import fetch_install

await fetch_install("example.com/your-package.whl")

import your_package
# omit
`);

2.5 Inspect Shared Object (for Debugging)

Fot inspect, numpy is also required, and you can install with pip install exodide[inspect].

python3 -m exodide.inspect your-package/your-module.so

Currently, exodide.inspect module prints dylink / dylink.0 custom section, which are used for metadata of Wasm dynamic link.

3. LICENSEs

We utilize other projects and these codes obey their original lisences. We distribute patched header files of CPython and NumPy, too.

• CPython: https://www.python.org/
  • cpython directory
  • PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
• NumPy: https://numpy.org/
  • numpy directory and script/code_generators directory
  • BSD 3-Clause
• Pyodide: https://pyodide.org/en/stable/
  • pyodide directory
  • MPL-2.0
• Others (exodide original codes)
  • MIT

4. Build exodide

git clone --recursive --depth 1 https://github.com/ymd-h/exodide.git
cd exodide

make

pip install .