build: WASM/WASI Nix flake environment and build helper scripts#151
Open
angerman wants to merge 6 commits intostable-ghc-9.14from
Open
build: WASM/WASI Nix flake environment and build helper scripts#151angerman wants to merge 6 commits intostable-ghc-9.14from
angerman wants to merge 6 commits intostable-ghc-9.14from
Conversation
angerman
force-pushed
the
stable-ghc-9.14
branch
from
05e1a0c to
7968d7c
Compare
Switch from manual shell.nix management to using the ghc-wasm-meta flake which provides a properly integrated wasi-sdk with libffi-wasm. Benefits: - wasi-sdk from ghc-wasm-meta includes libffi-wasm pre-integrated - Maintained by the GHC WASM team - Reproducible builds via flake.lock - No manual download/patching required The wasi-sdk package from ghc-wasm-meta is used directly, eliminating the need for manual libffi-wasm installation scripts. See: https://gitlab.haskell.org/haskell-wasm/ghc-wasm-meta
…ride When wasi-sdk is in buildInputs, nix's mkShell automatically sets CC and LD to the WASM cross-compiler tools. This breaks any native compilation during the build (e.g. stage1 deriveConstants, configure scripts). The wasi-sdk is now accessed exclusively via WASI_SDK_DIR in shellHook, and the cross-compiler tools are only available via named wrappers (wasm32-wasi-clang, wasm-ld, etc.) in PATH.
GHC's WASM LLVM backend uses llc/opt/llvm-as to compile LLVM IR to WASM assembly. Using nixpkgs LLVM 18 tools with wasi-sdk's clang 21 assembler causes version mismatch errors: 'error: .size directive ignored for function symbols' Fix by creating wrappers that use wasi-sdk's LLVM 21 versions of llc, opt, and llvm-as, which are compatible with wasi-sdk's clang 21.
- Export WASM_EXTRA_LIB_DIRS and WASM_EXTRA_INCLUDE_DIRS pointing to the wasi-sysroot, so the Makefile can pass --extra-lib-dirs and --extra-include-dirs to cabal for WASM RTS compilation. - Fix help text: the Makefile default CABAL=_build/stage0/bin/cabal is correct; no CABAL= override is needed. Just run: make stage2 / stage3-... - Keep ghc-wasm-meta unpinned (latest): with TNTC disabled (see previous commit), info table symbols are pure DATA, not WASM FUNCTION types. The new LLVM's .size restriction on function symbols does not apply. Using the latest ghc-wasm-meta is safe and provides the full wasi-sdk with wasi-sysroot already cached from previous builds.
build-wasm-make.sh: - Switch from nix-shell to nix develop (flake-based environment) - Remove explicit CABAL= override; Makefile default _build/stage0/bin/cabal is correct and stage0 cabal is auto-built from the repo's pinned source build-wasm-on-linux0.sh: - Fix rsync to exclude both '.git' file (git worktree reference) and '.git/' directory; prevents broken worktree path from being synced - After rsync, re-initialize a proper git repo on the remote with git init + git add -A + git commit; required for nix develop to evaluate flake.nix via git+file:// URL - Update prerequisites: only Nix with flake support needed (no ghcup)
- Document switch from nixpkgs pkgsCross.wasi32 to wasi-sdk 24.0 - Explain nixpkgs LLD bug that prevented linking - Document libffi incompatibility with WASI (Emscripten-only) - Clarify wrapper scripts approach for LD_LIBRARY_PATH propagation - Add reference to libffi source showing Emscripten requirement
angerman
force-pushed
the
build/wasm-nix-environment
branch
from
1d99ecb to
2e88edb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the Nix flake development environment and remote build helper scripts
needed to build the WASM/WASI cross-compiler (
wasm32-unknown-wasi-ghc)on a Linux machine from a macOS workstation.
This is the build infrastructure companion to PR #142 (
fix/wasm-remove-pic-guards),kept separate so the compiler/RTS fixes can be reviewed independently.
What's included
flake.nix/flake.lock: Nix flake that pulls inghc-wasm-metaforwasi-sdk(clang, wasm-ld, wasi-sysroot) and exportsWASM_EXTRA_LIB_DIRS/WASM_EXTRA_INCLUDE_DIRSfor the build.Uses wasi-sdk's LLVM 21 tools (
llc,opt,llvm-as) so versions match.Does not add
wasi-sdktobuildInputs(would overrideCC/LDand break the native stage2 build).
build-wasm-make.sh: wrapper that entersnix developand runsmake stage3-wasm32-unknown-wasiwith the correct environment.build-wasm-on-linux0.sh: syncs the local source tree to a remoteLinux machine via rsync (excluding
_build/,.git, generated autoconffiles) and triggers a full build there. Uses
--no-deleteto preserveautoconf-generated files on the remote that are absent from a git worktree.
USAGE.md: step-by-step build instructions covering prerequisites,build invocation, and how to test the resulting
wasm32-unknown-wasi-ghc.Test plan
nix develop .opens a shell withwasm32-unknown-wasi-clangon PATHnix develop . -c make stage3-wasm32-unknown-wasicompletes on x86_64-linuxwasm32-unknown-wasi-ghc /tmp/hello.hs -o /tmp/hello.wasm && wasmtime /tmp/hello.wasmprintsHello WASM