Spack

Spack#

Working with Spack environments and package specs.

Pin a setup’s Spack specs#

List the Spack packages a setup needs and let knit build them.

APIs: knit_with_spack_specs, knit_register_setup, KNIT_SETUP_PREFIX

For the common “just install these packages” case, list the Spack specs a setup needs with knit_with_spack_specs. Knit synthesizes a minimal spack.yaml and, before the setup body runs, installs the specs into a Spack environment and activates it — so the tools are on PATH / LD_LIBRARY_PATH when you build into KNIT_SETUP_PREFIX. The Spack it uses is knit’s own, installed under .knit/ and isolated from any Spack you have on the machine (see Run Spack directly and Isolate Spack from your personal config); declaring a Spack-backed setup makes bootstrap provision that Spack automatically:

@setup "juliaenv" "Build and install julia-fractal from source."
@with_spack_specs "cmake" "libpng"
@with_optional "ref:string" "v1.1.0" "git ref to build (tag, branch, or commit)."
_juliaenv_setup() {
    # The Spack environment declared above is already built and activated here, so
    # cmake and libpng are on PATH / LD_LIBRARY_PATH. Everything we install goes
    # under KNIT_SETUP_PREFIX --- the private directory Knit created for this setup.
    local ref
    ref="$(knit_get_parameter "ref" "$@")"

    git clone "https://github.com/knit-sh/julia-fractal-example.git" \
        "${KNIT_SETUP_PREFIX}/src"
    git -C "${KNIT_SETUP_PREFIX}/src" checkout "${ref}"

    # Configure, build, and install into the setup prefix. No MPI is present in
    # the environment, so CMake builds the serial binary. RPATH_USE_LINK_PATH
    # bakes the libpng location into the binary so it also runs on a machine with
    # no system libpng.
    cmake -S "${KNIT_SETUP_PREFIX}/src" -B "${KNIT_SETUP_PREFIX}/build" \
        -DCMAKE_INSTALL_PREFIX="${KNIT_SETUP_PREFIX}" \
        -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON
    cmake --build "${KNIT_SETUP_PREFIX}/build"
    cmake --install "${KNIT_SETUP_PREFIX}/build"

    # Put the installed binary on the PATH of every command that depends on this
    # setup. knit_setup_env_prepend records a composable line, so each dependent
    # command keeps its own PATH and gains this entry.
    knit_setup_env_prepend PATH "${KNIT_SETUP_PREFIX}/bin"
}
@done

knit_register_setup "juliaenv" _juliaenv_setup "Build and install julia-fractal from source."
knit_with_spack_specs "cmake" "libpng"
knit_with_optional "ref:string" "v1.1.0" "git ref to build (tag, branch, or commit)."
_juliaenv_setup() {
    # The Spack environment declared above is already built and activated here, so
    # cmake and libpng are on PATH / LD_LIBRARY_PATH. Everything we install goes
    # under KNIT_SETUP_PREFIX --- the private directory Knit created for this setup.
    local ref
    ref="$(knit_get_parameter "ref" "$@")"

    git clone "https://github.com/knit-sh/julia-fractal-example.git" \
        "${KNIT_SETUP_PREFIX}/src"
    git -C "${KNIT_SETUP_PREFIX}/src" checkout "${ref}"

    # Configure, build, and install into the setup prefix. No MPI is present in
    # the environment, so CMake builds the serial binary. RPATH_USE_LINK_PATH
    # bakes the libpng location into the binary so it also runs on a machine with
    # no system libpng.
    cmake -S "${KNIT_SETUP_PREFIX}/src" -B "${KNIT_SETUP_PREFIX}/build" \
        -DCMAKE_INSTALL_PREFIX="${KNIT_SETUP_PREFIX}" \
        -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=ON
    cmake --build "${KNIT_SETUP_PREFIX}/build"
    cmake --install "${KNIT_SETUP_PREFIX}/build"

    # Put the installed binary on the PATH of every command that depends on this
    # setup. knit_setup_env_prepend records a composable line, so each dependent
    # command keeps its own PATH and gains this entry.
    knit_setup_env_prepend PATH "${KNIT_SETUP_PREFIX}/bin"
}
knit_done

Each argument is a full Spack spec, so you can pin versions and variants the same way you would on the spack install command line (for example knit_with_spack_specs "hdf5@1.14 +mpi" "fftw"). The concrete manifest and lockfile are captured as provenance outputs on the setup’s table. For finer control over the environment, use a full manifest with knit_with_spack_env instead.

Use a full Spack manifest#

Describe a setup’s Spack environment with a complete spack.yaml.

APIs: knit_with_spack_env, knit_register_setup

When you need full control over the environment — versions, variants, compilers, views, package preferences — describe it with a complete spack.yaml manifest via knit_with_spack_env. Point it at a file checked in next to the experiment:

# Point @with_spack_env at a spack.yaml checked in next to the experiment.
# The manifest gives full control (versions, variants, compilers, views).
@setup "libs" "Build dependencies from a spack.yaml."
@with_spack_env "spack.yaml"
_libs_setup() {
    # cmake/hdf5/... from the manifest are on PATH here; build into the prefix.
    echo "building in ${KNIT_SETUP_PREFIX}"
}
@done

# Point @with_spack_env at a spack.yaml checked in next to the experiment.
# The manifest gives full control (versions, variants, compilers, views).
knit_register_setup "libs" _libs_setup "Build dependencies from a spack.yaml."
knit_with_spack_env "spack.yaml"
_libs_setup() {
    # cmake/hdf5/... from the manifest are on PATH here; build into the prefix.
    echo "building in ${KNIT_SETUP_PREFIX}"
}
knit_done

Or, when the manifest is short and specific to the experiment, inline it with a here-doc (any redirected stdin works). The manifest is read at registration time, so it must actually be redirected — an interactive terminal or an empty manifest is rejected rather than left to block:

# Or inline the manifest with a here-doc when it is short and experiment-specific.
@setup "libs-inline" "Build deps from an inline manifest."
@with_spack_env <<'EOF'
spack:
  specs:
    - hdf5@1.14 +mpi
    - fftw
  view: true
EOF
_libs_inline_setup() {
    echo "building in ${KNIT_SETUP_PREFIX}"
}
@done

# Or inline the manifest with a here-doc when it is short and experiment-specific.
knit_register_setup "libs-inline" _libs_inline_setup "Build deps from an inline manifest."
knit_with_spack_env <<'EOF'
spack:
  specs:
    - hdf5@1.14 +mpi
    - fftw
  view: true
EOF
_libs_inline_setup() {
    echo "building in ${KNIT_SETUP_PREFIX}"
}
knit_done

A setup may declare at most one Spack environment, and knit_with_spack_env is mutually exclusive with the knit_with_spack_specs sugar (which funnels through it). As with specs, the concrete manifest and lockfile are captured as provenance outputs on the setup’s table. The Spack itself is knit’s own, installed under .knit/ at bootstrap time — declaring any Spack-backed setup makes bootstrap detect that the experiment needs Spack and provision it, and bootstrap --spack provisions (and pins) it explicitly.

Build an MPI environment and provide a launcher#

Ask Spack for an MPI provider in a setup and expose it as the run launcher.

APIs: knit_register_setup, knit_with_spack_specs, knit_provides_launcher

Add mpi (or a concrete provider like mpich or openmpi) to a setup’s knit_with_spack_specs so Spack provisions an MPI implementation as part of the environment. Calling knit_provides_launcher then advertises that Spack-built MPI as the launcher, so knit run can place ranks even on a machine that has no MPI of its own:

# Ask Spack for MPICH and expose it as the launcher. Activating the Spack
# environment (which knit does before the body runs) puts mpicc / mpirun on PATH,
# and @provides_launcher advertises that MPI to "knit run" --- so the body has
# nothing left to build.
@setup "mpienv" "Provide an MPI implementation via Spack."
@with_spack_specs "mpich"
@provides_launcher
_mpienv_setup() {
    : # nothing to do: Spack already installed and activated MPICH
}
@done

# Ask Spack for MPICH and expose it as the launcher. Activating the Spack
# environment (which knit does before the body runs) puts mpicc / mpirun on PATH,
# and @provides_launcher advertises that MPI to "knit run" --- so the body has
# nothing left to build.
knit_register_setup "mpienv" _mpienv_setup "Provide an MPI implementation via Spack."
knit_with_spack_specs "mpich"
knit_provides_launcher
_mpienv_setup() {
    : # nothing to do: Spack already installed and activated MPICH
}
knit_done

Because knit activates the Spack environment before the setup body runs, the MPI compilers and launcher (mpicc / mpirun) are already on PATH — the body here has nothing left to do. In a real setup this is exactly where the environment pays off: a downstream build (say a CMake find_package(MPI)) now finds the Spack MPI and produces a parallel binary, with no MPI-specific logic in the setup itself.

The provided launcher is detected once at setup-build time and frozen into the setup’s .activate.sh (recorded as the __mpi_launcher__ provenance output). It sits below a machine’s own launcher in precedence, so a profile’s launcher still wins when one exists; it only fills the gap on a machine that offers none.

Run Spack directly#

Drive the experiment’s private Spack with the knit spack wrapper.

APIs: spack, bootstrap

knit spack is a thin wrapper that forwards every argument verbatim to the experiment’s private Spack, so you can inspect and drive it exactly as you would a system Spack — without it leaking onto your PATH:

$ ./exp.sh spack find            # what is installed
$ ./exp.sh spack spec zlib       # how a spec would concretize
$ ./exp.sh spack --help          # Spack's own help, forwarded

Because the wrapper forwards --help too, ./exp.sh spack --help shows Spack’s help rather than knit’s. The private Spack is provisioned automatically the first time a setup declares a Spack environment; run knit spack before any such setup and it fails with a hint to bootstrap with --spack (or to add a Spack-backed setup). To provision it eagerly and pin its version at bootstrap, see the Pin the Spack version recipe in the Bootstrap category.

Isolate Spack from your personal config#

Knit’s Spack ignores ~/.spack and site config so the environment stays reproducible.

APIs: spack

The Spack that knit provisions is deliberately walled off from your personal and site-wide Spack configuration, so an experiment concretizes the same way no matter whose machine or account it runs on. Knit exports two environment variables for its Spack:

  • SPACK_DISABLE_LOCAL_CONFIG=true — Spack ignores both the system scope and your ~/.spack user scope.

  • SPACK_USER_CONFIG_PATH pointing at a directory inside the experiment (.knit/.spack) — any user-scope config Spack reads or writes lives with the experiment, not in your home directory.

The upshot is that what the experiment builds is determined only by the specs/manifest in the script plus knit’s own config under .knit — both of which travel with the experiment — so a stray package preference, mirror, or compiler default in ~/.spack can neither leak into nor silently change the build. To customize the environment, put the settings in the manifest (see Use a full Spack manifest) or edit knit’s own scope with knit spack config, rather than in ~/.spack.

Pin the Spack version#

Provision a specific Spack (and package repo) git ref at bootstrap.

APIs: bootstrap

Knit provisions a private Spack automatically the first time a setup declares a Spack environment, defaulting to the latest Spack release. Pass --spack to pin a specific Spack git ref (a tag, branch, or commit) instead, and --spack-packages to pin the package repository ref:

$ ./exp.sh bootstrap \
    --spack v0.22.1 \
    --spack-packages v2024.11.0

Both take a git ref as their value; an empty value (the default) uses the latest release. Passing --spack also forces Spack to be provisioned now rather than lazily on first use.