Profiles

Selecting a machine profile at bootstrap and reading its scheduler, launcher, and hardware fields.

Select a machine profile

Pass bootstrap –profile to prepopulate scheduler, launcher, and hardware defaults from a shared machine description.

APIs: bootstrap, profile:list, profile:show

A machine profile is a shared JSON description of a cluster — its scheduler, MPI launcher, default queue, and per-node core/GPU counts. Selecting one at bootstrap saves you from spelling those out by hand and keeps every experiment on that machine consistent.

See which profiles are available, and inspect one before committing to it:

$ ./exp.sh profile list
  polaris   [github] Argonne Polaris (PBS, MPICH).
  perlmutter  [github] NERSC Perlmutter (Slurm, Cray MPICH).
$ ./exp.sh profile show anl/polaris

profile list and profile show work before bootstrap, so you can browse first. A profile spec is a shorthand like anl/polaris, a URL, or a local file — handy for a machine that is not in the public index.

Then bootstrap with it:

$ ./exp.sh bootstrap --profile anl/polaris --project m1234

The profile’s values become the defaults for --scheduler, --launcher, --default-cpus-per-node, and the rest, so an explicit flag is only needed to override one. The resolved profile is frozen into the experiment’s metadata at bootstrap, so the experiment stays reproducible even if the shared profile later changes; profile show on a bootstrapped experiment prints that frozen copy. Read individual fields from a script with Read a profile field in a script.

Read a profile field in a script

Call knit_get_profile_field with a jq path to read any field from the bootstrapped machine profile inside a command body.

APIs: knit_get_profile_field, knit_list_profiles

Once a profile is frozen at bootstrap (see Select a machine profile), a command body can read any of its fields with knit_get_profile_field and a jq path expression:

Shorthand

@command "sizing" "Report placement derived from the machine profile."
_sizing() {
    local scheduler cores
    scheduler="$(knit_get_profile_field '.scheduler.type')"
    cores="$(knit_get_profile_field '.hardware.cores_per_node')"
    printf 'scheduler=%s cores_per_node=%s\n' \
        "${scheduler:-unknown}" "${cores:-unknown}"
}
@done

Long form

knit_register "sizing" _sizing "Report placement derived from the machine profile."
_sizing() {
    local scheduler cores
    scheduler="$(knit_get_profile_field '.scheduler.type')"
    cores="$(knit_get_profile_field '.hardware.cores_per_node')"
    printf 'scheduler=%s cores_per_node=%s\n' \
        "${scheduler:-unknown}" "${cores:-unknown}"
}
knit_done

Each call takes a jq path into the profile JSON (.scheduler.type, .hardware.cores_per_node, .launcher.command, …) and prints the value, with string quotes stripped. When the experiment was bootstrapped without a profile, or the field is absent, it prints nothing — so default it in the caller, as above with ${scheduler:-unknown}. This lets a command adapt its placement to the machine it is running on without hard-coding cluster details.

To enumerate the profiles knit knows about from a script (the same list the profile list command prints), call knit_list_profiles; pass true to include hidden ones.

Read the platform name in a script

Call knit_platform_name to get the human-facing name of the machine the experiment was bootstrapped for.

APIs: knit_platform_name

Every bootstrapped experiment records a platform name — a short, human-facing label for the machine it runs on. Read it from a command body with knit_platform_name:

Shorthand

@command "where" "Report the platform this experiment runs on."
_where() {
    printf 'platform=%s\n' "$(knit_platform_name)"
}
@done

Long form

knit_register "where" _where "Report the platform this experiment runs on."
_where() {
    printf 'platform=%s\n' "$(knit_platform_name)"
}
knit_done

The name is set at bootstrap. Pass it explicitly with --platform:

$ ./exp.sh bootstrap --platform mymachine

When you bootstrap with a machine profile (see Select a machine profile) and omit --platform, the name defaults to the profile’s own name — e.g. anl/polaris for the Polaris profile — so a profile-based experiment self-identifies without extra flags. knit_platform_name prints an empty string when no platform was recorded.

Anatomy of a machine profile

A walkthrough of a real profile (ALCF Polaris) — its scheduler, launcher, hardware, modules, and Spack sections.

APIs: bootstrap, profile:show

A profile is a JSON document describing one machine. Here is the profile knit ships for ALCF Polaris:

{
    "name": "anl/polaris",
    "description": "ALCF Polaris — HPE Cray EX, 32 cores + 4× NVIDIA A100 per node",
    "scheduler": {
        "type": "pbs",
        "command": "qsub",
        "default_queue": "prod",
        "default_args": ["-l filesystems=home:eagle"],
        "queues": {
            "debug":         { "min_nodes": 1,  "max_nodes": 2,   "min_walltime": "00:05:00", "max_walltime": "01:00:00", "default_walltime": "01:00:00" },
            "debug-scaling": { "min_nodes": 1,  "max_nodes": 10,  "min_walltime": "00:05:00", "max_walltime": "01:00:00", "default_walltime": "01:00:00" },
            "prod":          { "min_nodes": 10, "max_nodes": 496, "min_walltime": "00:05:00", "max_walltime": "24:00:00", "default_walltime": "01:00:00" },
            "preemptable":   { "min_nodes": 1,  "max_nodes": 10,  "min_walltime": "00:05:00", "max_walltime": "72:00:00", "default_walltime": "01:00:00" }
        }
    },
    "launcher": {
        "type": "pals",
        "command": "mpiexec"
    },
    "hardware": {
        "cores_per_node": 32,
        "gpus_per_node": 4
    },
    "modules": ["PrgEnv-gnu/8.6.0"],
    "spack": {
        "packages": {
            "cray-mpich": {
                "externals": [
                    {
                        "spec": "cray-mpich@9.0.1",
                        "prefix": "/opt/cray/pe/mpich/9.0.1/ofi/gnu/12.3",
                        "modules": ["PrgEnv-gnu/8.6.0", "libfabric/2.2.0rc1", "cray-mpich/9.0.1"]
                    }
                ],
                "buildable": false
            },
            "libfabric": {
                "externals": [
                    {
                        "spec": "libfabric@2.2.0",
                        "modules": ["libfabric/2.2.0rc1"]
                    }
                ],
                "buildable": false
            },
            "mpi": {
                "require": ["cray-mpich"]
            }
        }
    }
}

Reading it top to bottom:

• description — a one-line human summary, shown by profile list and profile show.

• _hide — when true, keeps the profile out of profile list unless --hidden is passed. Use it for internal or work-in-progress profiles; the profile still resolves by name.

• scheduler — the batch system. type (pbs here) selects the backend knit uses to write directives and submit; command is the submit binary; default_queue and default_args become the experiment’s defaults; and queues records each queue’s node/walltime bounds so submit can validate a request against the queue before it reaches the scheduler.

• launcher — how MPI ranks are launched. type (pals) picks the launcher backend for knit run, and command is its binary (mpiexec).

• hardware — cores_per_node and gpus_per_node, used as the defaults for whole-node placement (--default-cpus-per-node, --gpus-per-node) and readable from a script with knit_get_profile_field.

• modules — environment modules to module load before anything runs, so setups and jobs start from the machine’s supported programming environment (here PrgEnv-gnu).

• spack — Spack configuration merged into every Spack environment knit builds. Its keys are Spack config sections (packages here); knit writes them back verbatim, so this is just Spack config. Polaris registers the vendor MPI (cray-mpich) and libfabric as non-buildable externals pointing at the system install, and makes the virtual mpi package require cray-mpich — so any setup that asks Spack for mpi gets the tuned Cray stack instead of building its own.

The whole document is frozen into the experiment’s metadata at bootstrap, so bootstrap --profile anl/polaris gives every command on that machine the same scheduler, launcher, hardware, and Spack defaults. To author your own, see Write a machine profile.

Write a machine profile

Guidance for profile authors — capture only what the experiment cannot do itself, and keep the Spack section to vendor packages.

APIs: bootstrap, knit_with_spack_specs

Writing a profile for a new machine (see Anatomy of a machine profile for the format) comes down to one principle: a profile captures only what an experiment script cannot do for itself. A profile is machine truth — which scheduler, which launcher, how many cores and GPUs per node, which vendor libraries the site provides — not a snapshot of everything installed on the login node.

The spack section is where this discipline matters most. Restrict it to vendor packages: the software that is tied to the machine and must come from the site, not from a portable build. On a typical HPC system that is the tuned MPI, the network fabric library, the GPU runtime, and the compilers — exactly what the Polaris profile lists (cray-mpich, libfabric). Register those as non-buildable externals:

"cray-mpich": {
    "externals": [ { "spec": "cray-mpich@9.0.1", "prefix": "/opt/cray/pe/mpich/9.0.1/ofi/gnu/12.3" } ],
    "buildable": false
}

and, where the machine provides a specific implementation of a virtual package, point the virtual at it ("mpi": { "require": ["cray-mpich"] }) so every setup’s knit_with_spack_specs "mpi" resolves to the vendor stack.

Do not add externals for software the experiment can build itself. Even if the platform ships cmake, git, or a recent python, leave them out of the profile: an experiment’s own Spack environment can build those portably, and pinning them to a site path only makes the experiment less reproducible and harder to move to the next machine. A profile that lists CMake as an external is describing the login node, not the machine. The test for each Spack entry is: could a setup build this correctly on its own? If yes, it belongs in a setup’s knit_with_spack_specs, not in the profile.

The same restraint applies to modules: list only the modules needed to reach the vendor software (the programming environment, an MPI module), not a personal working set. Keep profiles small, vendor-focused, and durable — they should change only when the machine does.