Private API

The private API consists of names with one or two leading underscores, grouped below by the source file that defines them. It is internal to Knit and may change at any time, without notice and without a compatibility guarantee.

ai.sh

Functions

int _knit_ai_chat_request()

Build an OpenAI chat-completions request from the given model, messages array, and optional tools array, POST it to <base_url>/chat/completions, and print the raw response JSON on stdout for the caller to parse.

The API key is passed to curl through a mode-600 config file (never a -H flag), so it never appears in the process argv (ps) or in any trace. Only a redacted form of the request is traced. A provider error (.error in the response) is surfaced through the logging system and turned into a fatal; the request body is never dumped.

Parameters:

• base_url -- [in] Endpoint base URL (without a trailing /chat/completions).

• api_key -- [in] Resolved API key (kept local; never logged).

• model -- [in] Model id.

• messages_json -- [in] JSON array of chat messages.

• tools_json -- [in] Optional JSON array of tool definitions (omitted when empty).

int _knit_ai_default_system_prompt()

Print the default system prompt for ai ask: a short explanation of knit, the instruction to answer about this experiment using only the read-only tools, and a compact seeded summary of the available commands. Overridden wholesale by ai ask --system.

int _knit_ai_describe_summary()

Print a compact one-line-per-command summary of the whole command tree ("- <full command path>: <description>"), used to seed the model's first turn so it need not call knit_describe just to learn what commands exist. Built from the machine-readable describe --format json --compact output. Best-effort: a failure prints nothing (the model can still call the tools for detail).

int _knit_ai_dispatch_tool()

Route a model-requested tool call to its handler and print the (truncated) result. Tool arguments arrive as a JSON object string and are parsed with _knit_jq -r. Only allowlisted tools run; an unknown name yields an error string. Handler output (including any fatal from the underlying knit surface, e.g. an unknown job id) is captured so it becomes the tool result rather than aborting the agentic loop.

Before running the handler, the recording-suppression state is cleared (unset KNIT_DISABLE_RECORDING, reset _KNIT_RECORDING_SUPPRESSED) so a recordable tool-command would record exactly as if the user had run it directly. The ai command's own non-recording state never leaks into the commands it drives.

Parameters:

• name -- [in] The tool name (e.g. "knit_describe").

• args_json -- [in] JSON object string of the tool's arguments.

int _knit_ai_loop()

Run the agentic tool-calling loop for ai ask. Seeds the conversation with the system prompt and the user question, then repeatedly POSTs the accumulating message array (with the read-only tool schema) to the provider. On each turn: the assistant message is appended; if it carries no tool calls it is the final answer (printed and the loop returns); otherwise every requested tool is dispatched and its result appended as a tool-role message before the next turn. The message array is grown with _knit_jq --argjson so JSON types stay intact.

Stops after max_iterations rounds with a warning if no final answer is reached.

Parameters:

• base_url -- [in] Resolved endpoint base URL.

• api_key -- [in] Resolved API key (passed straight to the request helper).

• model -- [in] Resolved model id.

• question -- [in] The user's natural-language question.

• system_prompt -- [in] The system prompt to seed the conversation with.

• max_iterations -- [in] Hard cap on tool-call rounds.

• raw -- [in] "true" to print the raw final message JSON instead of its text.

• verbose -- [in] "true" to stream each tool call and result to stderr.

Returns:

0 when a final answer is produced, 1 on hitting the iteration cap.

int _knit_ai_resolve_config()

Resolve the provider access configuration for an ai call. Reads the ai.* metadata keys, indirect-expands the stored env-var names to their values (never eval), and applies precedence/fallbacks:

• api key: value of the env var named by ai.api_key_env (required).

• base url: value of the env var named by ai.base_url_env if set, else the ai.base_url literal, else _KNIT_AI_DEFAULT_BASE_URL.

• model: the model_override argument if non-empty, else the value of the env var named by ai.model_env, else the ai.model literal.

Fatals (with a hint pointing at bootstrap --ai-*) when the provider is unconfigured, the API key env var is empty/unset, or no model can be resolved. The resolved API key is returned only through the caller-named output variable; it is never logged, traced, or echoed.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the resolved API key.

• __knit_ret2 -- [out] Name of the variable to hold the resolved base URL.

• __knit_ret3 -- [out] Name of the variable to hold the resolved model id.

• model_override -- [in] Optional model id that takes precedence over metadata.

int _knit_ai_sql_is_readonly()

Shared read-only SQL guard for the knit_db_query tool and (in a later milestone) ai query. A statement is considered read-only when its leading keyword is one of SELECT / WITH / EXPLAIN / PRAGMA and it contains no write keyword (INSERT / UPDATE / DELETE / DROP / ALTER / CREATE / REPLACE / ATTACH) as a whole word anywhere. The second check catches piggy-backed writes such as "SELECT 1; DROP TABLE runs" that pass the leading-keyword test.

Word boundaries are handled by uppercasing the statement and replacing every non-word character with a space, then matching the space-padded keyword; this avoids relying on the non-POSIX \b regex escape and never misfires on a column named e.g. created_at (the underscore keeps it one token).

Parameters:

• sql -- [in] The SQL statement to check.

Returns:

0 if the statement is read-only, 1 otherwise.

int _knit_ai_store_config()

Write the provider-access configuration to the metadata table as ai.* key/value pairs, through the same metadata store path used elsewhere. This is the writer used by bootstrap's --ai-* options to store a full config in one shot (update mode writes each key on its own).

Only env-var names and non-secret defaults are stored; no API key ever reaches the database.

Parameters:

• api_key_env -- [in] Name of the env var holding the API key.

• base_url_env -- [in] Name of the env var holding the endpoint base URL.

• model_env -- [in] Name of the env var holding the model id.

• base_url -- [in] Literal fallback base URL.

• model -- [in] Literal fallback model id.

• overwrite -- [in] "true" to overwrite existing keys, anything else to fail on a duplicate key.

int _knit_ai_tool_db_query()

Tool handler: run a read-only SQL query against the experiment database and print the result (aligned columns with a header). The statement must pass _knit_ai_sql_is_readonly; a rejected statement returns an error string (fed back to the model) rather than being run. The query always runs on the read path (_knit_sqlite3, never _knit_sqlite3_write), so it can never mutate the DB.

Parameters:

• sql -- [in] The SQL statement to run.

int _knit_ai_tool_describe()

Tool handler: structured introspection of the command tree, as YAML. Optional only (comma-separated command list) and recursive narrow/expand the scope.

Parameters:

• only -- [in] Optional comma-separated command list (colon form, e.g. "a,b:c").

• recursive -- [in] "true" to also include the selected commands' subcommands.

int _knit_ai_tool_help()

Tool handler: the --help text for one specific command. The command name may contain spaces for a nested command (e.g. "job show stdout"); it is split into words before being passed to knit.

Parameters:

• command -- [in] The command whose help to show (space-separated for nesting).

int _knit_ai_tool_job_output()

Tool handler: a recorded job's captured stdout/stderr or its generated batch script, via the job show <stream> subcommands (there is no top-level knit stdout command).

Parameters:

• id -- [in] The job UUID.

• stream -- [in] One of "stdout" (default), "stderr", or "script".

int _knit_ai_tool_metadata_show()

Tool handler: all experiment metadata. Everything in the metadata table is safe to send (only env-var names and non-secret config, never secrets).

int _knit_ai_tools_schema()

Print the OpenAI tools[] JSON array describing the read-only tool set exposed to the model. There is deliberately no command-execution tool: the model can inspect the experiment but cannot run experiment commands or mutate anything.

int _knit_ai_truncate()

Print the given text, cut to _KNIT_AI_TOOL_OUTPUT_MAX_BYTES with an explicit "…(truncated)" marker appended when it is longer. Used to bound every tool result handed back to the model.

Parameters:

• text -- [in] The text to bound.

Variables

String _KNIT_AI_DEFAULT_BASE_URL

Literal fallback base URL used when no base-url env var resolves to a value. Kept in sync with the same default hard-coded on bootstrap's --ai-base-url option (src/boostrap.sh loads before this file, so it cannot reference this variable at registration time).

String _KNIT_AI_TOOL_OUTPUT_MAX_BYTES

Approximate ceiling on the size of a single tool result handed back to the model. A tool whose output exceeds this is cut to the budget and marked with an explicit "…(truncated)" line, so a huge describe or query result can't blow the context window silently. Measured in characters (== bytes for ASCII, which covers knit's introspection output).

app.sh

Functions

int _knit_app_before_cb()

Before-callback installed on every app subcommand by knit_register_app. Verifies that KNIT_JOB_PREFIX is set: an app only runs inside a job (launched by the run dispatcher and forwarded to each rank by the launcher), so its absence means the app command was invoked directly. Mirrors _knit_setup_before_cb / _knit_job_before_cb.

int _knit_run()

Entry point for the run CLI command: the user-facing dispatcher that launches an app across a subset of the surrounding job's allocation. A run has no meaning outside a job (the job supplies the node list the launcher places ranks on), so this fails fast when invoked outside one (KNIT_JOB_PREFIX unset), analogous to how job and setup commands reject direct invocation.

It validates the app name and its arguments, resolves the placement options into a concrete (procs, procs-per-node, hosts) triple, resolves the launcher backend, and execs the per-rank worker (_run) under that launcher. The app runs in the context of the job's directory (there is no separate run directory); the run's UUID serves only as its id in the runs table. Its exit status is the launcher's.

Usage:

./exp.sh run [placement-opts...] -- app-name [args...]   (inside a job)

int _knit_run_checksum_inputs()

Verify existence of, and hash, every checksummed file/directory input of an app once, on the login side, before it is launched. A missing input then fails fast — before an allocation is used or ranks are spawned — and each input is hashed exactly once rather than once per rank. The bare digests are returned by name (param -> 64-hex); the dispatcher forwards them to rank 0 through the launcher environment (KNIT_CHECKSUM_<param>) so the app's row records the same digest without any rank touching the filesystem. Existence is checked for every file/directory input, even one that opted out of the digest with --no-checksum; only a checksummed input is hashed.

Parameters:

• out_name -- [out] Name of the associative array to fill (param -> bare 64-hex).

• subcmd -- [in] Mangled app command name (run:<app>).

• ... -- [in] The app's expanded invocation arguments.

int _knit_run_checksum_outputs()

Verify existence of, and hash, every checksummed file/directory output of an app once, on the login side, after the launcher returns — never on a rank. An app's body runs on every rank, but only rank 0 records the per-app row, and it records only the output paths (the checksum columns are left empty by the M4 output hook, which no-ops in an app worker). Hashing here, after launch, keeps it off every rank and out of every measured duration: hashing on rank 0 would keep the launcher blocked while the other ranks idle at finalize, inflating the run's measured wall-clock.

Rank 0's per-app row is found through the provenance graph: the "run -> run:app" call edge (source_id is this run's UUID) points at the row's id. For each checksummed output, the recorded path is read back from that row, its existence verified (a missing output on a successful run is fatal), the digest computed, and the row's companion "<param>_checksum" column updated. An output left with no value, or one that opted out with --no-checksum, is skipped. When nothing was recorded (recording disabled, or no such row) this is a no-op.

Parameters:

• subcmd -- [in] Mangled app command name (run:<app>).

• app_name -- [in] The app name (its table is named after it).

• run_uuid -- [in] This run's UUID (source of the provenance edge to the row).

int _knit_run_normalize_mpi_env()

Normalize the launcher-native MPI environment into the launcher-agnostic KNIT_MPI_* variables the app body reads, and export them so every subprocess of the rank inherits them:

KNIT_MPI_RANK rank in MPI_COMM_WORLD KNIT_MPI_SIZE size of MPI_COMM_WORLD KNIT_MPI_LOCAL_RANK node-local rank

Each is taken from the first launcher that set it, by precedence (OpenMPI -> MPICH/PMI -> Slurm srun -> PALS -> Flux), falling back to a single rank-0 / size-1 process when none are present (the none backend). Called once per rank by the worker, before forwarding to the app body.

int _knit_run_resolve_placement()

Resolve the three placement options (--procs, --procs-per-node, --hostnames) into a concrete (procs, procs-per-node, hosts) triple, stored by name into the caller's associative array (keys: procs, procs-per-node, hostnames). procs is always set; procs-per-node may be left empty (the launcher's default distribution applies); hostnames is always set to a comma-separated list.

Let A be the job's allocated unique hosts (knit_job_hostnames), a = |A|; c the per-node core count (node_ncpus metadata, may be unknown); n = --procs, p = --procs-per-node, H = --hostnames (k = |H|). Resolution:

1. Hosts. If --hostnames is given, each entry must be one of A (fatal otherwise); else H defaults to A. 2/3. procs / procs-per-node. If both n and p are given they are used as-is (see the consistency checks). If only n is given, H is respected; if H was explicit, p is derived as n/k. If only p is given, n is derived as p*k. If neither is given, n = c*k with p = c when c is known, else n = k (one rank per node) with a warning.

2. Consistency (fatal on conflict): n and p given => require np==0 (kreq=n/p); with explicit H require k==kreq, else require kreq<=a and take the first kreq hosts of A. n given without p but with explicit H => require nk==0 (p=n/k).

Parameters:

• out_name -- [out] Name of the associative array to fill.

• ... -- [in] The dispatcher's invocation arguments (read via knit_get_parameter).

int _knit_run_worker()

Hidden per-rank worker executed once per rank under the launcher. The run dispatcher translates placement once and execs <launcher> [flags] ./exp.sh _run -- <app> [app opts]; this worker runs on every rank, then forwards to the app body by re-entering the command machinery as run:<app> (both tokens are non-"--", so this routes to the app, never back to the run dispatcher — there is no recursion).

Each rank normalizes the launcher-native MPI environment into KNIT_MPI_* (via _knit_run_normalize_mpi_env) before forwarding to the app, and every rank but rank 0 sets _KNIT_RECORDING_SUPPRESSED so a run's outputs and per-app row are recorded exactly once.

By default an app has no setup of its own: it inherits the surrounding job's setup environment (forwarded by the launcher). An app may still declare knit_with_setup to explicitly depend on and re-source a setup (see knit_register_app).

Variables

AssociativeArray _KNIT_APPS

Associative array mapping registered app names to 1. Used to validate that an app name passed to knit run is known. Mirrors _KNIT_JOBS for jobs.

String _KNIT_RUNS_TABLE

Name of the table recording every run: the app launched, the requested placement, and the launcher. The row id is the run's own UUID. The parent job and the rank-0 per-app row are linked through the provenance graph — the "submit:<job> -> run" and "run -> run:<app>" call edges — not through a stored column or a shared id (each mints its own distinct UUID).

artifact.sh

Functions

int _knit_artifact_cardinality_phrase()

Store the human-readable cardinality phrase for a quantifier in the caller-named variable: "zero or more" for "*", "one or more" for "+", and the empty string for a scalar (empty quantifier). Shared by every describe format and --help so a collection reads the same way everywhere.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the phrase (empty if scalar).

• quant -- [in] Cardinality quantifier ("*", "+", or empty).

int _knit_artifact_check_lower_bound()

Enforce the lower bound of every variadic "+" output the command declared: a "name:kind+" collection must bind at least one member. Called at record time, after the body (see _knit_artifacts_record_sql), so a "+" output the body left unbound is fatal and names the offending output. A "*" output that bound nothing is allowed (a fan-out that produced nothing is not an error), and a scalar output has no lower bound here (its at-most-once rule lives in knit_artifact). Iterates the command's declared artifacts set and consults the per-invocation binding stash.

Parameters:

• cmd -- [in] Mangled command name of the producer.

• demangled -- [in] Demangled producer name, used in the fatal message.

int _knit_artifact_ensure_table_cb()

knit_done callback (installed by _knit_artifact_require_table) that gives an artifact-producing command a table when it declared none, so its invocation is recorded and can be the source of a "produced" edge. A command that declared its own table with knit_with_table is left untouched. The auto-created table takes the command's own (demangled) name, mirroring knit_with_table's default, is registered in the names map, and is set up immediately (deferred to first use when the experiment is not yet bootstrapped, exactly like knit_with_table).

Parameters:

• cmd -- [in] Mangled command name.

int _knit_artifact_is_bound()

Return 0 when the command bound the named artifact at least once in this invocation, 1 otherwise. Consults the per-invocation binding stash filled by knit_artifact (KNIT_CMD<cmd>_artifact_bound, keyed by normalized name).

Parameters:

• cmd -- [in] Mangled command name.

• name -- [in] Normalized artifact name.

Returns:

0 if bound, 1 otherwise.

int _knit_artifact_kind_type()

Look up the physical type ("file" or "directory") backing an artifact kind and store it in the caller-named variable. Returns non-zero when the kind is not registered, leaving the variable untouched. Consulted by the output and input artifact directives to resolve a declared "name:kind" annotation to its physical type.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the physical type.

• kind -- [in] The artifact kind name.

Returns:

0 if the kind is registered, 1 otherwise.

int _knit_artifact_parse_kind()

Split an optional trailing cardinality quantifier off the kind portion of an artifact spec (the text after "name:"). A single trailing "*" (zero or more) or "+" (one or more) marks the artifact as a variadic collection; a kind with no quantifier is scalar. The bare kind (quantifier removed) and the quantifier character ("" for scalar, "*", or "+") are stored in the caller's variables. Shared by knit_with_output_artifact and knit_with_input_artifact so the syntax and its validation cannot drift between the two directions.

A kind carrying more than one trailing quantifier (e.g. "file**" or "file*+") is fatal, so a typo is caught at registration rather than silently dropped. The bare kind is not validated here (each caller validates it against the kind registry as before).

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the bare kind.

• __knit_ret2 -- [out] Name of the variable to hold the quantifier ("", "*", "+").

• raw_kind -- [in] The kind portion of the spec (may carry a quantifier).

• context -- [in] Caller name, used in the fatal message.

int _knit_artifact_require_table()

Guarantee that the command currently being registered will record an invocation row, so a "produced" edge from it has a source. Called by knit_with_output_artifact. The row comes from the command's table; if the command declares its own table (knit_with_table) nothing more is needed, so this only pushes a knit_done callback (_knit_artifact_ensure_table_cb) that creates a default table when none was declared. The callback is pushed at most once per command, however many artifacts it declares.

int _knit_artifact_resolve_path()

Resolve a user-supplied <linked-path> into the artifact entry's absolute location and its artifacts-relative form, enforcing containment on the entry's OWN location, not its target. A relative <linked-path> is taken against the artifacts root; an absolute one is used as given. The parent directory is resolved to its real path (so a symlink in the parent chain is followed), while the final component is kept verbatim (a symlink there is not followed, so a symlink artifact stays inside the artifacts root even when its target is elsewhere). Containment holds when that real parent is the artifacts root or a directory below it.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the absolute entry path.

• __knit_ret2 -- [out] Name of the variable to hold the artifacts-relative path.

• root -- [in] The resolved (absolute) artifacts root.

• linked_path -- [in] The user-supplied <linked-path>.

Returns:

0 on success (contained); 1 when <linked-path> is outside the root.

int _knit_artifact_root()

Store the resolved artifact root — the directory under which artifacts live — in the caller-named variable. Reads the verbatim artifact_path from the metadata table (falling back to "artifacts" when unset, for robustness) and resolves it against the experiment root via _knit_resolve_experiment_path. Mirrors _knit_setup_root / _knit_job_root / _knit_resource_root.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved artifact root.

int _knit_artifacts_create_table()

Create the artifacts table if it does not already exist. Each row is one produced artifact: a stable identity ("id", a uuidv7, the target of the "produced" edge), the artifacts-relative "path" (UNIQUE, since an on-disk entry is write-once), the declared "name" the producer used, the physical "type" ("file" or "directory"), the semantic "kind" the producer declared ("csvfile", "rundir", ...; "file"/"directory" for a bare builtin), the content "checksum" ("sha256:<hex>"), and a "result" flag (1 when declared --result, else 0). The schema is fixed here rather than derived from a command's declared outputs (unlike a per-command table), so its column order and the UNIQUE constraint on "path" are explicit. Called at bootstrap alongside the metadata and provenance tables.

int _knit_artifacts_ensure_table()

Ensure the artifacts table exists before an artifact row is written, creating it lazily on first use. A freshly bootstrapped experiment already has the table (created at bootstrap); a database bootstrapped before this feature shipped does not, so ensuring it here lets a new invocation record artifacts rather than failing. The create is idempotent and runs at most once per process, guarded by _KNIT_ARTIFACTS_TABLE_ENSURED. Mirrors _knit_prov_ensure_table.

int _knit_artifacts_path_recorded()

Return 0 when the given artifacts-relative path already has a row in the artifacts table (recorded by an earlier invocation, possibly in a previous run), 1 otherwise. knit_artifact consults this to turn the write-once "path" UNIQUE constraint into a clear fatal before the INSERT is built, rather than leaking sqlite3's raw "UNIQUE constraint failed: artifacts.path" at record time. It is a read, so no lock is taken. It reports "not recorded" (returns 1) when the experiment is not bootstrapped or the artifacts table does not yet exist, since no row can be recorded then.

Parameters:

• path -- [in] Artifacts-relative path to look up.

int _knit_artifacts_record_sql()

Build (into the caller's variable, without executing) the SQL that records every artifact the given command bound during this invocation: one artifacts row plus one "producer --produced--> artifact" edge per stashed binding. The result is spliced into the producing row's own transaction (see _knit_db_record_invocation) so the row, its "call" edge, and its produced artifacts are written atomically.

Each binding was stashed by knit_artifact keyed on the artifacts-relative path; the name and content digest come from that stash, while the physical type ("file" or "directory"), the semantic kind, and the "result" flag are recovered from registration state (the fileparam marker holds the physical type, the 3<name>_type field holds the kind, and the results set holds the flag). A fresh uuid identifies each artifacts row (the target of its "produced" edge). The caller's variable is left empty when the command bound no artifact.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the built SQL.

• cmd -- [in] Mangled command name of the producer.

• producer_id -- [in] Row id of the producing invocation (the edge source).

• producer_name -- [in] Demangled producer name (the edge source_name).

int _knit_artifacts_row_sql()

Build (print, without executing) the INSERT for one artifacts row: one produced artifact, with its uuid "id" (the target of the "produced" edge), its artifacts-relative "path" (UNIQUE), the declared "name", the physical "type" ("file" or "directory"), the semantic "kind" ("csvfile"/"rundir"/...; "file"/"directory" for a bare builtin), the content "checksum", and the "result" flag. The text columns are single-quoted and escaped; "result" is emitted as a bare 0/1 integer (any value other than "1" becomes 0). Meant to be composed with a "produced" edge (see _knit_produced_edge_sql) into one transaction at record time, mirroring how _knit_prov_edge_sql composes into _knit_db_record_invocation.

Parameters:

• id -- [in] UUID of the artifact row.

• path -- [in] Artifacts-relative path (UNIQUE).

• name -- [in] Declared artifact name.

• type -- [in] Physical type: "file" or "directory".

• kind -- [in] Semantic kind the producer declared.

• checksum -- [in] Content digest of the resolved target ("sha256:<hex>").

• result -- [in] "1" for a declared result, else 0.

int _knit_input_artifact_after_cb()

After-callback installed by knit_with_input_artifact on the consuming command, one per declared input-artifact parameter. It records a "used_by" edge from each consumed artifact to the command itself. It runs as an after-callback (not the before-callback that validates the artifact) because the consumer's frame is on the executing stacks only from push time onward, so the consumer's resolved row id — the edge target — is available here but not in the before-callback (this mirrors _knit_resource_dep_after_cb).

The declared parameter name, required kind, verify flag, and cardinality quantifier are bound at registration time and carried as the first four arguments (the parameter name and the quantifier are used here); the trailing arguments are the command's own runtime arguments, scanned for the parameter value. A missing value is silently skipped: the before-callback already fataled on it, so this is only reached with a valid value.

A scalar input records the single edge for its one value. A variadic input resolves its comma-separated value with the shared list resolver (the same one the before-callback validated against) and records one "used_by" edge per resolved element, so every consumed member joins the lineage; an empty "*" list resolves to nothing and records no edge.

Parameters:

• param -- [in] Normalized name of the input-artifact parameter to read.

• kind -- [in] Required artifact kind (unused; kept for argument symmetry).

• verify -- [in] Checksum opt-in flag (unused; kept for argument symmetry).

• quant -- [in] Cardinality quantifier ("*"/"+"), empty for a scalar input.

int _knit_input_artifact_before_cb()

Before-callback installed by knit_with_input_artifact on the consuming command, one per declared input-artifact parameter. Before the command body runs it resolves the parameter value and validates it. A scalar input carries a single artifacts-relative path, validated as one element. A variadic input ("*"/"+") carries a comma-separated list, resolved (split then glob, see _knit_input_artifact_resolve_list) into elements that are validated in turn; the first bad element is fatal and names which element. A missing value for a scalar input is fatal, an empty "+" list is fatal, and an empty "*" list validates trivially (the body simply receives no element). Each element is checked by _knit_input_artifact_validate_one (existence + kind, plus an opt-in checksum). Returning normally lets the command proceed; a fatal aborts it before the body runs.

The declared parameter name, required kind, verify flag, and cardinality quantifier are bound at registration time and carried as the first four arguments; the trailing arguments are the command's own runtime arguments, scanned for the parameter value. Mirrors _knit_resource_dep_before_cb.

Parameters:

• param -- [in] Normalized name of the input-artifact parameter to read.

• kind -- [in] Artifact kind the resolved artifact(s) must carry.

• verify -- [in] "1" to re-verify the recorded checksum, empty to skip.

• quant -- [in] Cardinality quantifier: "" scalar, "*" zero or more, "+" one or more.

int _knit_input_artifact_param_kind()

Store the artifact kind a parameter was declared with in the caller-named variable, or the empty string when the parameter is an ordinary parameter (not declared through knit_with_input_artifact). Reads the per-parameter marker (KNIT_CMD<cmd>input_artifact) that knit_with_input_artifact records, so describe and --help can annotate an input-artifact parameter with its required kind from the registration tables alone (no database read). The parameter name must be normalized, as it is stored in the parameter sets. Mirrors _knit_resource_param_type.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the artifact kind (empty if none).

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter name.

int _knit_input_artifact_quantifier()

Store the cardinality quantifier a consumed artifact parameter was declared with in the caller-named variable: "*" (zero or more), "+" (one or more), or the empty string for a scalar parameter. Reads the per-parameter marker (KNIT_CMD<cmd>input_artifact_variadic) that knit_with_input_artifact records, so describe and --help can mark a collection distinctly from a scalar from the registration tables alone (no database read). The parameter name must be normalized, as it is stored in the parameter sets.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the quantifier (empty if scalar).

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter name.

int _knit_input_artifact_record_used_by_edge()

Record a "used_by" provenance edge from a consumed artifact to a consuming invocation. The edge's source is the artifacts row (its id resolved by path, its node name the "artifacts" label — the same node the "produced" edge points at, so lineage joins as A --produced--> artifact --used_by--> B); its target is the consumer. Delegates the gated write to _knit_record_used_by_edge, so it records nothing when recording is disabled, on a suppressed rank, before bootstrap, when the target does not participate in the graph, or when no row is found at the path (empty id -> no edge). Mirrors _knit_resource_record_used_by_edge.

Parameters:

• value -- [in] Artifacts-relative path of the consumed artifact.

• target_cmd -- [in] Mangled command name of the consumer (the edge target).

• target_id -- [in] Resolved row id of the consumer (the edge target).

int _knit_input_artifact_resolve_list()

Resolve a variadic input-artifact value into an array of artifacts-relative paths. The raw value is a comma-separated list; each element is resolved in two steps: split on the comma, then expand. An element that holds a glob metacharacter ("*", "?", "[") is expanded with bash pathname expansion relative to the artifacts root, under nullglob, so a pattern matching nothing yields no element (never the literal pattern) and recursive "**" stays disabled; the artifacts-root prefix is then stripped so each match is an artifacts-relative path (the form the artifacts row stores). An element with no metacharacter is taken verbatim. An empty element (from an empty value or a stray comma) is skipped. The result keeps element order, and within one globbed element bash's sorted order, and is de-duplicated first-seen, so overlapping patterns do not record a path twice. Word splitting is disabled during expansion, so a root or element that holds a space stays intact. Shared by the input-artifact before-callback, after-callback, and knit_input_artifact_paths, so validation, edges, and the body's view cannot drift.

Parameters:

• __knit_ret -- [out] Name of the array variable to fill with relative paths.

• raw -- [in] The raw parameter value (comma-separated list of paths).

• root -- [in] The resolved (absolute) artifacts root.

int _knit_input_artifact_validate_one()

Validate one resolved input-artifact element against the recorded artifacts table: the path must name a recorded artifacts row (an unbootstrapped experiment has recorded none, so a missing row is treated as "no artifact"), the recorded kind must equal the declared kind (the physical type is implied by the kind, so it needs no separate check), and — when --verify-checksum was declared — the recomputed digest of the resolved entry must equal the recorded checksum. Any failure is fatal and names the offending element, so the first bad element of a variadic list aborts the command before its body runs. Shared by the scalar and variadic paths of _knit_input_artifact_before_cb.

Parameters:

• kind -- [in] Artifact kind the element must carry.

• verify -- [in] "1" to re-verify the recorded checksum, empty to skip.

• value -- [in] The element's artifacts-relative path.

int _knit_output_artifact_quantifier()

Store the cardinality quantifier a produced artifact was declared with in the caller-named variable: "*" (zero or more), "+" (one or more), or the empty string for a scalar artifact. Reads the per-name marker (KNIT_CMD<cmd>artifact_variadic<name>) that knit_with_output_artifact records, so describe and --help can mark a collection distinctly from a scalar from the registration tables alone (no database read). The name must be normalized, as it is stored in the artifacts set.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the quantifier (empty if scalar).

• cmd -- [in] Mangled command name.

• name -- [in] Normalized artifact name.

int _knit_register_artifact()

Add an artifact of the command being registered to its artifacts set: a name that refers to a file or directory that lives under the artifacts root and is bound at runtime with knit_artifact. Membership in this set is what marks a name as an artifact rather than an ordinary value output; it drives how the command is described and, later, how knit_artifact validates the name. An artifact is not an output column, so it is kept out of the outputs set.

The per-command KNIT_CMD<cmd>_artifacts set is created as associative on first use (knit_register does not create it, since not every command has an artifact).

Only meaningful in a command context; a call with no command being registered is a no-op.

Parameters:

• name -- [in] The declared (un-normalized) artifact name.

Variables

String _KNIT_ARTIFACTS_TABLE

Name of the framework-owned table that records one row per produced artifact (a first-class provenance node, like "jobs" and "runs"). A "produced" edge in the provenance table links a producing invocation to the artifact row whose id it names. A direct names-map entry (see below) resolves "artifacts" as a graph node label; the table has no owning command.

String _KNIT_ARTIFACTS_TABLE_ENSURED

Set to "1" once the artifacts table has been ensured in this process (see _knit_artifacts_ensure_table), so the idempotent CREATE runs at most once per run. Mirrors _KNIT_PROV_TABLE_ENSURED.

AssociativeArray _KNIT_ARTIFACT_KINDS

Registry mapping each artifact kind name to its backing physical type ("file" or "directory"). A kind is an artifact's semantic category (e.g. "csvfile", "rundir"); the physical type drives existence checks and checksumming. Seeded with the builtin kinds "file" and "directory" (each backed by its own type) and the "dir" alias (backed by "directory"), so a bare "name:file" / "name:directory" declaration needs no knit_register_artifact call and re-registering one of them is fatal. Extended by knit_register_artifact and consulted by _knit_artifact_kind_type. Declared global (-g) so it survives being sourced from within a function (as the bats tests do). Mirrors _KNIT_RESOURCES / _KNIT_ENUMS.

AssociativeArray _KNIT_ARTIFACT_KIND_DESCRIPTIONS

Companion registry mapping each artifact kind name to its one-line description, shown by knit describe / --help. Populated alongside _KNIT_ARTIFACT_KINDS; the builtin kinds carry a short default description.

boostrap.sh

Functions

int _knit_bootstrap()

int _knit_bootstrap_check_prerequisites()

Verify the external tools that Knit needs at run time but does not itself provision are present on PATH, and fail bootstrap early (with a clear message) when one is missing. Currently checks sha256sum, which content checksums for file/directory parameters and outputs depend on. Uses _knit_command_path so it is stubbable in tests.

int _knit_bootstrap_dir_has_subdir()

Return 0 when the given directory holds at least one subdirectory, non-zero otherwise (including when the directory itself is absent). Used by update-mode --resource-path relocation to refuse moving a resource root that already holds fetched resource instances (each is a subdirectory).

Parameters:

• dir -- [in] Directory to scan.

int _knit_bootstrap_jobs_recorded()

Return 0 when the "jobs" table exists and holds at least one row, non-zero otherwise. Used by update-mode --job-path relocation to refuse moving a job root that already holds submitted jobs. The table is created lazily on the first submission, so an experiment that never submitted a job has no "jobs" table at all; a missing table reports "no jobs".

int _knit_bootstrap_on_exit()

Clean up on exit if bootstrap did not complete successfully.

int _knit_bootstrap_relocate_path()

Update-mode handler for a path-root option (--setup-path, --job-path, --resource-path). When the option is typed and its value differs from the stored one, relocate the root — but only when the current root holds no artifacts of its kind, since setups, jobs, and resources are not movable:

• setup: no user setup exists (the builtin "default" does not count).

• job: no job row is recorded in the "jobs" table.

• resource: no resource instance directory exists under the root.

A non-empty root is a hard stop (fatal). On an allowed relocation it stores the new value, recreates the builtin "default" setup under the new root (setup kind only), and removes the old root directory (which, for the setup kind, still holds the old "default", so a plain rmdir would not suffice). An untyped option or a value equal to the stored one is a no-op.

Parameters:

• kind -- [in] One of "setup", "job", "resource".

• new_value -- [in] Resolved value typed for the option.

• ... -- [in] Raw argument tokens of this invocation (see _KNIT_INVOCATION_RAW_ARGS), used to tell a typed option from a defaulted one.

Returns:

0 when the root was relocated, 1 when nothing changed.

int _knit_bootstrap_update()

Run bootstrap in update mode on an experiment that is already bootstrapped. It changes only the options the user typed on this call and leaves the database and every other setting untouched; it installs no destructive exit trap. A .knit/ directory without a database is malformed and fatals with a hint to remove it.

It handles the free-to-update options (project, platform, account, default walltime, default cpus-per-node, default nodefile, scheduler, launcher), the AI provider options (--ai-*), each written per key so one AI field can change without clearing the rest, and the path-root options (--setup-path, --job-path, --resource-path), which relocate only when the current root is empty of its kind. It also handles the constrained options: --spack/--spack-packages (add Spack when absent, or re-provision on a changed ref only while no environment is built) and --profile (a changed profile is out of scope and fatals), and the tooling options: --ignore-system-sqlite/--ignore-system-jq (rebuild a system-symlinked tool from source) and --knit-cypher-to-sql-version/--knit-cypher-to-sql-url (re-provision knit-cypher-to-sql on a changed version or URL). When no handled option was typed, it reports that there is nothing to update and succeeds.

Parameters:

• raw_args -- [in] Name of an array holding the raw, pre-expansion argument tokens (the caller's copy of _KNIT_INVOCATION_RAW_ARGS).

• ... -- [in] The expanded argument list (defaults injected), as read with knit_get_parameter.

int _knit_bootstrap_update_meta()

Update-mode handler for a "free to update" bootstrap option: one stored as metadata with no constraint (project, platform, account, ...). When the option was typed on this bootstrap call, overwrite its stored value with "metadata store --force" and report that a change was made; when the option was not typed, do nothing and keep the stored value.

Parameters:

• opt -- [in] Option name as typed, without the leading "--" (e.g. "project").

• key -- [in] Metadata key to write (e.g. "__project__").

• value -- [in] Resolved value to store for that option.

• ... -- [in] Raw argument tokens of this invocation (see _KNIT_INVOCATION_RAW_ARGS), used to tell a typed option from a defaulted one.

Returns:

0 when the option was typed and its value written, 1 otherwise.

int _knit_bootstrap_update_profile()

Update-mode handler for --profile. Changing the machine profile is out of scope for the re-runnable bootstrap: a typed profile that differs from the stored one is a hard stop (fatal), while a typed profile equal to the stored one (or an untyped profile) is a no-op. The stored profile is the resolved profile label recorded at first bootstrap (profile).

Parameters:

• profile -- [in] Value typed for --profile.

• ... -- [in] Raw argument tokens of this invocation (see _KNIT_INVOCATION_RAW_ARGS), used to tell a typed option from a defaulted one.

Returns:

1 when there is nothing to do (a differing profile fatals instead).

int _knit_bootstrap_warn_absolute_root()

Warn (non-fatal) when a bootstrap path-root option was given an absolute value. Absolute setup/job roots pin the experiment to this machine's filesystem and hurt reproducibility on another machine or by another user; a relative value resolves against the experiment root and stays portable. A no-op for relative values. Factored out of _knit_bootstrap so it can be unit-tested directly.

Parameters:

• label -- [in] The option name to name in the warning (e.g. "--setup-path").

• value -- [in] The value the user supplied for that option.

int _knit_highlight_if_not_bootstrapped()

Highlight predicate (see knit_highlight_if) for the builtin "bootstrap" command: return 0 ("highlight") while the experiment has not been bootstrapped, non-zero once it has. This bolds "bootstrap" in the root "--help" on a fresh checkout — the one command to run first — and leaves it plain afterwards.

Parameters:

• cmd -- [in] The demangled command name (unused; the predicate is state-only).

int _knit_is_bootstrapped()

Return 0 if the experiment has been bootstrapped (i.e. _KNIT_PREFIX exists), 1 otherwise.

The positive result is cached in _KNIT_IS_BOOTSTRAPPED so that repeated calls within the same session avoid redundant filesystem accesses. The negative result is never cached: the directory may be created at any moment by a bootstrap invocation in the same session.

Variables

String _KNIT_IS_BOOTSTRAPPED

Cache for _knit_is_bootstrapped(). Empty means "not yet checked"; "1" means the positive result has been confirmed and the filesystem need not be re-checked.

String _KNIT_PREFIX

Prefix directory for Knit's local installation.

bundle.sh

Functions

int _knit_bundle()

Body of "knit bundle": pack the experiment into one shippable archive. It reads the --output and --zip options, resolves the experiment root, computes the default output path when none is given, collects the minimal default contents, and drops any path the writer cannot pack. With --dry-run it prints the planned contents (a tree, or a flat list with --list, and sizes with --size) and writes nothing; otherwise it writes the archive. The command is read-only: it declares no table and takes knit_without_provenance, so it records no row and writes no provenance edge.

Parameters:

• ... -- [in] The command invocation arguments.

int _knit_bundle_auto_require()

Record a path that Knit adds to the bundle on the experiment's behalf (see _KNIT_BUNDLE_AUTO_REQUIRES). Called by other directives — knit_with_spack_env in its file form — not by the user. Like knit_bundle_requires it stores the string verbatim and MUST NOT touch the filesystem, so it stays safe when the experiment script is re-sourced on a compute node during job re-entry.

Parameters:

• path -- [in] A file path, meant relative to the experiment script's directory.

int _knit_bundle_collect()

Fill a caller-named array with the archive contents, each as a path relative to the experiment root. The default set carries what makes the experiment readable and re-runnable: the experiment script, the knit.sh framework beside it, the pruned .knit directory (the database only), the user-declared required files, each setup's small manifest files, each job's logs and scripts, and the declared artifacts. It leaves out the bulky, regenerable parts: the provisioned tools under .knit, each setup's built environment, and the fetched resources.

The options nameref selects what to include. Each key holds "true"/"false" except include_resources, which holds a comma-separated resource-name list:

• no_knit drop the knit.sh framework file;

• no_db drop .knit/knit.db;

• no_job_logs drop each job's .stdout / .stderr;

• no_job_scripts drop each job's .job.sh / .job.id;

• include_job_content also pack the user content of each job directory;

• no_artifacts drop the artifacts tree;

• include_all_resources pack every fetched resource;

• include_resources pack the named fetched resources.

The experiment script, knit.sh, .knit/knit.db, and the user-declared required files are added unconditionally, so a later prune step can warn about a missing one. Every framework-enumerated path (setup, job, artifact, resource) is added only when it exists on disk (a regular path or a symlink, even a broken one), so an absent optional manifest — a non-Spack setup has no spack.yaml, say — is skipped silently rather than warned about. Glob expansion of the required entries and the portability validations come in a later milestone.

Parameters:

• __knit_ret -- [out] Name of the array to fill with relative paths.

• opts -- [in] Name of an associative array of include/exclude options.

• root -- [in] The absolute experiment root.

int _knit_bundle_default_output()

Compute the default archive path when the user gives no --output, and store it in the caller-named variable. The name is "./<project>-bundle" plus the format extension. The project name comes from the metadata table (key project); it falls back to the experiment script name without its ".sh" extension when the metadata holds no project. The extension is ".zip" for the zip format and ".tar.gz" otherwise.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the default path.

• fmt -- [in] The archive format ("zip" or "tar").

int _knit_bundle_escapes_root()

Test whether a relative path escapes its root, i.e. a ".." component takes it above the starting directory. Pure string logic (no filesystem): it walks the components, treating "." and empty as no-ops, and reports an escape as soon as the depth would go negative.

Parameters:

• path -- [in] The relative path to test.

Returns:

0 if the path escapes the root, 1 otherwise.

int _knit_bundle_expand_requires()

Fill a caller-named array with the archive-relative paths from a list of required entries (see knit_bundle_requires), validating and expanding each one. Every entry is meant relative to the experiment root. A glob pattern is expanded with pathname expansion relative to the root; every match is added and a pattern that matches nothing draws a warning. A literal entry is added as-is.

The strict flag chooses how a portability problem is handled. In strict mode (the user's own knit_bundle_requires list) an absolute path, a ".." escape, and a missing literal are each a fatal that names the offending path, since the user typed it and can fix it. In lenient mode (paths Knit auto-required on the experiment's behalf) the same problems draw a warning instead: an absolute path is normalized under its basename (its real source recorded in the extern map, so its bytes still travel), and an escape or a missing literal is skipped.

Parameters:

• __knit_ret -- [out] Name of the array to fill with relative paths.

• root -- [in] The absolute experiment root.

• strict -- [in] "true" to fatal on a bad path, "false" to warn and continue.

• ... -- [in] The required entries.

int _knit_bundle_is_glob()

Test whether a path string is a glob pattern, i.e. it holds an unescaped pathname-expansion metacharacter ("*", "?", or "[").

Parameters:

• path -- [in] The path string to test.

Returns:

0 if the string is a glob pattern, 1 otherwise.

int _knit_bundle_print_list()

Print the planned bundle contents as a flat list, one path per line, each relative to the experiment root. This is the --dry-run --list form. When size mode is on, each line is prefixed with the path's size in bytes and a final TOTAL line gives the sum; the size of a symlink is its target's real size (du dereferences with -L), and the size of a directory is its recursive content.

Parameters:

• size_mode -- [in] "true" to annotate each path with its size in bytes.

• root -- [in] The absolute experiment root.

• ... -- [in] The relative paths to print.

int _knit_bundle_print_tree()

Print the planned bundle contents as a tree, like the layout in the design document. This is the default --dry-run form. It builds a parent-to-children map from the flat path list (synthesizing the intermediate directories that no entry names on its own), prints a root header from the archive label, and hands the rendering to _knit_bundle_render_tree. When size mode is on, each real entry is annotated with its size and a final TOTAL line gives the sum.

Parameters:

• size_mode -- [in] "true" to annotate each entry with its size in bytes.

• root -- [in] The absolute experiment root.

• label -- [in] The archive root name shown as the tree header.

• ... -- [in] The relative paths to render.

int _knit_bundle_prune_paths()

Fill a caller-named array with the candidate paths that exist on disk, dropping the ones the archive writer cannot pack. Each candidate is a path relative to the experiment root. A symlink whose target cannot be reached — a dangling link or a symlink loop — is skipped with a warning, so the writer neither packs a link that dangles on the reproducer's machine nor spins on a loop. A path that is simply absent is skipped with a warning too. Every surviving path is kept in the given order.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the surviving paths.

• root -- [in] The absolute experiment root.

• ... -- [in] The candidate relative paths.

int _knit_bundle_relpath()

Store, in the caller-named variable, an absolute path expressed relative to the experiment root. A path under the root has its "<root>/" prefix stripped, so it lands at the same place when the archive is unpacked.

A path outside the root is a portability hazard: a stored root (setup, job, artifact, or resource) that was bootstrapped as an absolute path does not relocate with the archive. When a canonical fallback name is given, the path is packed under that normalized relative name instead: a warning is emitted, the real absolute source is recorded in _KNIT_BUNDLE_EXTERN (so the prune, dry-run, and write steps can find its bytes), and the fallback name is returned. Without a fallback the outside path is returned unchanged.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the relative path.

• root -- [in] The absolute experiment root.

• abs -- [in] The absolute path to make relative.

• canonical -- [in] Optional normalized relative name for an outside path.

int _knit_bundle_render_tree()

Render one directory level of the bundle tree, then recurse into each child directory. It draws the box connectors ("├──", "└──", "│") from the child's position among its siblings. A node is shown with a trailing "/" when it has packed children or is a directory on disk. When size mode is on, a node that is a real bundle entry (named in the collected path list, so a leaf here) is annotated with its size in bytes and that size is added to the running total.

Parameters:

• kidsname -- [in] Name of the parent-to-children map (a value per line).

• realname -- [in] Name of the set of real bundle entries (leaf paths).

• prefix -- [in] The indentation drawn before this level's connectors.

• parentpath -- [in] The path whose children this call renders ("" is root).

• size_mode -- [in] "true" to annotate real entries with their size.

• root -- [in] The absolute experiment root.

• totalname -- [inout] Name of the running byte-total variable.

int _knit_bundle_source()

Store, in the caller-named variable, the real on-disk source of a collected relative path. For an ordinary in-tree path this is "<root>/<rel>". For a path that stands for out-of-tree content (its prefix is a key in _KNIT_BUNDLE_EXTERN) it is the recorded absolute source with the matching prefix swapped back in, so the bytes are found where they really live. Used by prune, the dry-run size report, and the archive writer.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the absolute source path.

• root -- [in] The absolute experiment root.

• rel -- [in] The collected relative path.

int _knit_bundle_warn_unselected_local()

Warn, for each fetched resource instance that is local-only (its type declared knit_with_local) and is not selected for the archive, that it will be left out. A local resource has no remote source, so — unlike a git or url resource — it cannot be re-fetched later; leaving it out should be a deliberate choice, not a silent omission. A resource is selected when --include-all-resources is set or its name is in the --include-resources list. The instance's type is read from its ".<name>.resource.type" sidecar marker, then mapped to the resource type's download method through the per-command _fetch_method marker.

Parameters:

• include_all -- [in] "true" when every resource is selected.

• include_list -- [in] Comma-separated selected resource names (may be empty).

int _knit_bundle_write_archive()

Write the archive from a list of paths, each relative to the experiment root, so the archive unpacks to the same relative tree anywhere. Symlinks are dereferenced: the target content travels in the archive, not the link, so the result is self-contained and relocatable. The tar format uses "-h" for this; the zip format dereferences by default (it stores the referenced file unless "-y" is given). The tar path reads the file list from a temporary file so a long list does not hit the command-line length limit; the zip path changes into the root first, because zip has no "-C" option.

A path may stand for content outside the experiment tree (an absolute stored root, see _KNIT_BUNDLE_EXTERN). Such paths cannot be packed relative to the root, so they are staged first: a symlink at the path's normalized relative name points to the real source, and the archive tool dereferences it into place. The in-tree paths pack straight from the root; only the outside paths need staging.

Parameters:

• fmt -- [in] The archive format ("zip" or "tar").

• output -- [in] The archive path to write.

• root -- [in] The absolute experiment root.

• ... -- [in] The relative paths to pack.

Returns:

Fatal when the archive tool is missing, no path is given, or the write fails; otherwise 0.

Variables

Array _KNIT_BUNDLE_AUTO_REQUIRES

Extra files Knit itself knows the experiment needs and adds without the user asking — currently the spack.yaml a setup names through knit_with_spack_env in its file form. Each entry is a path string recorded verbatim, in declaration order. Unlike _KNIT_BUNDLE_REQUIRES (the user's own list, validated strictly), these are handled leniently at bundle time: a path outside the tree draws a warning and is normalized rather than a fatal, since the user did not type it.

AssociativeArray _KNIT_BUNDLE_EXTERN

Maps a normalized in-archive relative path to the absolute on-disk source it stands for, for content that lives outside the experiment tree: an absolute stored root (a setup/job/artifact/resource root bootstrapped as an absolute path) or an auto-required file outside the tree. The key is where the content lands in the archive; the value is where its bytes really are. Collection fills it; the prune, dry-run, and write steps consult it (by prefix) to find the real source of a collected path. Reset at the start of every "knit bundle".

Array _KNIT_BUNDLE_REQUIRES

Extra files the experiment needs but Knit does not otherwise track (a config file, a small input, a post-processing script). Each entry is a path string recorded verbatim by knit_bundle_requires, in declaration order. The paths are meant to be relative to the experiment script's directory; validation and glob expansion happen later, when "knit bundle" runs, not at declaration time.

cli.sh

Functions

int _knit_arg_name()

Extract the normalized parameter name from a command-line token. The token may be given as "--name" or "--name=value"; the leading "--" and any "=value" suffix are stripped and hyphens are converted to underscores. This is the single place where the "=value" form is recognized, so registered commands and plain helper functions (via knit_get_parameter / knit_check_arguments) accept it consistently.

The caller is responsible for only passing tokens that start with "--"; bare values must not be passed, otherwise they may be mistaken for parameter names.

Parameters:

• token -- [in] A raw argument token starting with "--".

int _knit_arg_was_provided()

Report whether a named option appears among a command invocation's raw arguments. Used to tell an option the user typed apart from one that took its default: the CLI engine splices optional defaults and flag values into the argument list before a command body runs, so the body cannot see the raw command line except through the tokens _knit_invoke_command publishes in _KNIT_INVOCATION_RAW_ARGS.

The bare-flag form ("--name"), the separate-value form ("--name value"), and the joined form ("--name=value") are all recognized: only tokens starting with "--" are compared, by normalized name, so a following value token is never mistaken for the option. Hyphens and underscores in the option name are interchangeable. Scanning stops at a "--" token, so an option name that appears only after "--" (as an extra) does not count as provided.

Parameters:

• name -- [in] Option name, with or without the leading "--".

• ... -- [in] Raw argument tokens to scan (typically "${_KNIT_INVOCATION_RAW_ARGS[@]}").

int _knit_build_constraint_json()

Build a jq JSON object from an expanded argument list, using type metadata to emit integers/reals/booleans as JSON native types and all other values as strings. Each parameter may be given as "--name value" or "--name=value" (flags already converted to "true"/"false" by _knit_expand_command_arguments).

Parameters:

• cmd -- [in] Mangled command name (used for type lookups).

• ... -- [in] Expanded argument list.

int _knit_check_argument_type()

Validate that the value provided for a parameter conforms to the parameter's declared type. On mismatch the script stops with a fatal error; for enum types the message lists the accepted values. Parameters with no recorded type (e.g. framework-internal ones) are left unchecked.

Parameters:

• cmd -- [in] Command the parameter belongs to (mangled).

• demangled_cmd -- [in] Human-readable command name (used in messages).

• name -- [in] Parameter name (normalized).

• value -- [in] Value provided for the parameter.

int _knit_check_command_arguments()

Check that the arguments expected by the command are provided. This function will fail with a fatal error (i.e. the script will stop) if a required argument is not provided, if an argument provided does not match any expected, or if a value does not conform to its parameter's declared type.

Parameters:

• cmd -- [in] Name of the command (mangled).

• ... -- [in] Arguments to pass to the command.

int _knit_check_constraints()

Evaluate all --when constraints declared for a command against the provided arguments. For each constrained parameter:

• If the condition evaluates to true and the parameter is required but absent from the original (user-provided) arguments, a fatal error is raised.

• If the condition evaluates to false and the parameter is present in the original arguments, a fatal error is raised. Returns 0 immediately when the experiment is not bootstrapped (jq unavailable).

Parameters:

• cmd -- [in] Mangled command name.

• orig_ref -- [in] Name of bash array holding the original (pre-expansion) args.

• exp_ref -- [in] Name of bash array holding the expanded args.

int _knit_checksum_inputs()

For a command about to run, verify existence of every file/directory "input" parameter and, unless it opted out with --no-checksum, stash its digest before the timed body starts. The value comes from the expanded invocation arguments. A required input, or an optional input given a non-empty value, that does not exist is fatal; an optional input with no value is skipped (no existence check, no checksum). Existence is enforced for every file/directory input; hashing is skipped for a --no-checksum one. Hashing happens here, before the body, so a body that overwrites its own input cannot corrupt the recorded input digest and the hash time is excluded from the run.

In a launched app worker the inputs were already verified and hashed once by the run dispatcher before launch; no rank hashes. Rank 0 then recovers the forwarded digests from the environment (other ranks suppress recording), so this returns after that instead of touching the filesystem.

Parameters:

• cmd -- [in] Mangled command name.

• ... -- [in] The expanded invocation arguments.

int _knit_checksum_is_app_worker()

Report whether the command about to run is an app executing as a launched worker rank, rather than an ordinary single-process command. An app's inputs are verified and hashed once by the run dispatcher on the login side, before any rank is spawned; the ranks must not touch the filesystem for checksums. The context is an app (the "app" command type) re-entered under a live run (KNIT_RUN_ID, exported by the dispatcher into the launcher environment).

Parameters:

• cmd -- [in] Mangled command name.

int _knit_checksum_outputs()

After a command completes successfully, verify existence of every file/directory "output" and, unless it opted out with --no-checksum, stash its digest for the row write. The output value (the path) is read from the in-memory output store set by knit_output, else the output's declared default; an output left with no value is skipped (no existence check, no checksum). Existence is enforced for every file/directory output; hashing is skipped for a --no-checksum one. A declared output whose path does not exist is fatal. Called after the run duration has been captured, so hashing is excluded from the measured run time.

In a launched app worker no rank hashes outputs: rank 0 records the output paths with empty checksum columns, and the run dispatcher verifies existence and hashes them once after the launcher returns, off every measured duration. So this returns early there, leaving the checksum columns for the dispatcher.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_checksum_require_exists()

Verify that a checksummed file/directory value refers to an existing target, fataling with a direction-specific message if it does not. A "file" must be a regular file (-f), a "directory" must be a directory (-d). An input that does not exist is a broken precondition; an output missing on a successful completion is a broken postcondition — both are fatal, so a run never records a checksum for an artifact that was not there.

Parameters:

• demangled_cmd -- [in] Human-readable command name (for the message).

• direction -- [in] "input" or "output" (shapes the error message).

• name -- [in] Normalized parameter/output name (for the message).

• kind -- [in] "file" or "directory".

• value -- [in] The path to check.

int _knit_checksum_stash()

Record a computed digest for the currently-recording command by setting the companion "<param>_checksum" output value in the command's in-memory output store, so the normal row-recording machinery writes it. The digest is stored algorithm-prefixed ("sha256:<hex>"). This is the source-agnostic seam every path feeds: the non-app input hook, the non-app output hook, and (later) the app dispatcher all land a digest here for the same column writer.

Parameters:

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter/output name whose companion column to set.

• hex -- [in] The bare 64-hex digest (no algorithm prefix).

int _knit_checksum_stash_from_env()

Recover forwarded input digests for a launched app worker. The run dispatcher hashes each checksummed input once and forwards the bare digest to every rank through the launcher environment (KNIT_CHECKSUM_<param>). Rank 0 stashes those digests into its output store so its row records the same value the dispatcher computed, without any rank hashing. Other ranks suppress recording, so the stash is harmless there. An input with no forwarded digest (an absent optional, or one that opted out with --no-checksum) is skipped.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_command_check_usable()

Evaluate the usability predicates registered for a command (via knit_usable_if) in declaration order. On the first predicate that returns non-zero, set the reason output (nameref) to that predicate's parallel description and return 1. Return 0 if all predicates pass, or if the command declared none.

Each predicate is called as "<predicate> <demangled-cmd>" in the current shell (no subshell fork), so only its exit status is used. A predicate whose function does not exist is fatal: a usability guard that silently vanished would let an unusable command run.

Parameters:

• __knit_ret -- [out] Name of the variable to receive the failure reason (nameref).

• cmd -- [in] Command (mangled name) to check.

Returns:

0 if usable, 1 if a predicate failed (with the reason set).

int _knit_command_demangle()

Demangles a command, i.e. converts "command__1__subcommand__1__subcommand" back into "command:subcommand:subcommand".

Parameters:

• cmd -- [in] Command to demangle.

int _knit_command_display()

Rebuilds the human-readable name of a command from the registered spelling of each of its segments. Unlike _knit_command_demangle, which returns the canonical (underscore) identity, this restores the spelling the user wrote at registration (which may contain a hyphen), the same way a parameter keeps its registered spelling in "--help".

The mangled name is split into its "__1__"-separated prefixes; each prefix's stored display basename (KNIT_CMD<prefix>_display) is looked up and the basenames are joined with ":". A prefix with no stored basename (which should not happen, since a parent is always registered before its child) falls back to its canonical segment.

Parameters:

• cmd -- [in] Command to render (mangled name).

int _knit_command_get_last()

Takes a command in the form "aaa:bbb:ccc" or "aaa bbb ccc" or "aaa__1__bbb__1__cccc" and return the last command (e.g. "ccc" in all the cases above).

Parameters:

• __knit_ret -- [out] Name of the variable to hold the last command.

• cmd -- [in] Command name (colon/space/mangled).

int _knit_command_get_parents()

Takes a command in the form "aaa:bbb:ccc" or "aaa bbb ccc" or "aaa__1__bbb__1__cccc" and return the parent commands (e.g. "aaa:bbb" or "aaa bbb" or "aaa__1__bbb".

Parameters:

• __knit_ret -- [out] Name of the variable to hold the parent commands.

• cmd -- [in] Command name (colon/space/mangled).

int _knit_command_hidden()

The "--help" visibility test for a command. Return 0 (hidden) if the static _is_hidden boolean is true, or if any of the command's dynamic _hidden_pred predicates returns 0; return non-zero (shown) otherwise. Because knit_hidden and knit_hidden_if are mutually exclusive, at most one of the two arms is ever non-trivial for a given command.

Each dynamic predicate is called as "<predicate> <demangled-cmd>" in the current shell (no fork). A predicate whose function does not exist is a warning (not fatal): hiding is guidance, not access control, so a vanished predicate is treated as "no" (do not hide) and "--help" still renders.

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command should be hidden from "--help", non-zero otherwise.

int _knit_command_highlighted()

The "--help" highlight test for a command. Return 0 (highlight) if any of the command's dynamic _highlight_pred predicates returns 0; return non-zero (plain) otherwise, including when the command declares no highlight predicates.

Each predicate is called as "<predicate> <demangled-cmd>" in the current shell (no fork). A predicate whose function does not exist is a warning (not fatal): highlighting is cosmetic, so a vanished predicate is treated as "no" (do not highlight) and "--help" still renders.

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command name should be highlighted, non-zero otherwise.

int _knit_command_is_builtin()

Test whether a command is a framework builtin, i.e. it was marked with _knit_is_builtin during its registration.

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command is a builtin, 1 otherwise.

int _knit_command_is_usable_before_bootstrap()

Test whether a command is usable before bootstrap, i.e. it was marked with knit_usable_before_bootstrap during its registration.

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command is usable before bootstrap, 1 otherwise.

int _knit_command_is_wrapper()

Test whether a command is a wrapper, i.e. it was registered with knit_register_wrapper. A wrapper forwards its arguments verbatim to the underlying command and declares no parameters or outputs. Reads the command kind from the KNIT_CMD<cmd>_type field (see knit_register), which records the mutually-exclusive kind: "command" (plain, the default), "wrapper", "setup", "job", "app", or "resource".

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command is a wrapper, 1 otherwise.

int _knit_command_mangle()

Mangles a command, i.e. converts "command:subcommand:subcommand" into "command__1__subcommand__1__subcommand" so the name can be used in variable names. Also converts spaces into 1.

Hyphens are folded to underscores first, so a hyphen and an underscore in a command name name the same command (the canonical, Bash-safe identity uses the underscore), the same way hyphens and underscores are interchangeable in parameter names.

Parameters:

• cmd -- [in] Command to mangle.

int _knit_command_with_space()

Prints a mangled command (or a command with ":" in it) with spaces between subcommands.

Parameters:

• cmd -- [in] Command to print with spaces.

int _knit_decl_flag_present()

Return 0 if a bare declaration flag (e.g. "--no-checksum") appears among the remaining declaration arguments, 1 otherwise. Used to parse boolean flags such as --no-checksum that carry no value, so knit_get_parameter (which would read the following token as a value) is not suitable. Hyphens and underscores in the flag name are interchangeable, and anything from "--" onwards is ignored.

Parameters:

• flag -- [in] Flag name without the leading "--".

• ... -- [in] Declaration arguments to scan.

int _knit_execute_after_commands()

Evaluate the callbacks installed after a command. The callbacks are called with the calling command name (demangled) as context, as well as the list of parameters passed to the command.

Parameters:

• cmd -- [in] Command (mangled name) for which to execute the after callbacks.

• ... -- [in] Arguments of the command.

int _knit_execute_before_commands()

Evaluate the callbacks installed before a command. The callbacks are called with the calling command name (demangled) as context, as well as the list of parameters passed to the command.

A before-callback expresses a precondition for running the command (asserting an invocation context, activating a setup, building a Spack environment). If one fails (returns non-zero), the remaining callbacks are skipped and that status is returned so the caller (_knit_invoke_command) can abort the command rather than run its body against an unmet precondition.

Parameters:

• cmd -- [in] Command (mangled name) for which to execute the before callbacks.

• ... -- [in] Arguments of the command.

Returns:

0 if all callbacks succeeded, the first non-zero status otherwise.

int _knit_expand_command_arguments()

Adds optional arguments that are not provided in the arguments, and converts flags into --flag true or --flag false.

Parameters:

• name -- [in] Name of the command.

• ... -- [in] Arguments to pass to the command.

int _knit_find_flag()

This function takes a flag and checks if it appears in the remaining list of arguments, returning 0 if it does, 1 otherwise.

Example:

_knit_find_option "--help" aaa bbb ccc --help ddd

will return 0 because "--help" was found.

Parameters:

• flag -- [in] Flag to find.

• ... -- [in] List of arguments to search from.

Returns:

0 if the flag was found, 1 otherwise.

int _knit_format_option_alternatives()

Format the display spelling(s) of an option for a message. Options accept both hyphen and underscore spellings; this returns "--foo or --foo-bar" when the two spellings differ, and just "--foo" when they are identical (so a message about an option with no underscores does not read "--foo or --foo").

Parameters:

• __knit_ret -- [out] Name of the variable to hold the result.

• option -- [in] Parameter name (as declared).

int _knit_help_render_entry()

Render a single "--help" entry (an option row or a subcommand row) with an optional word-wrapped, hanging-indented description:

<head><first words of the description> <indent spaces><more words> ...

where <head> is the fully-formatted leading column (indentation, the padded option/subcommand name, and — for options — the "[annotation] " prefix) and continuation lines are indented to <indent> columns so the wrapped text forms a clean hanging indent under the description column.

Wrapping is applied only when a usable terminal width is known and the continuation column is wide enough; otherwise the entry falls back to a single line ("<head><description>"), byte-for-byte identical to the pre-wrapping output, so piped or redirected help stays unchanged.

Parameters:

• width -- [in] Terminal width in columns, or 0 for "no wrapping".

• head -- [in] Literal leading text printed before the description on the first line (may contain ANSI escape sequences).

• head_len -- [in] Display width of <head> (escape bytes excluded), i.e. the column at which the first-line description begins.

• indent -- [in] Number of spaces used to indent continuation lines.

• description -- [in] Description text, wrapped at word boundaries.

int _knit_hidden_if_not_usable_pred()

Internal hide predicate backing knit_hidden_if_not_usable. Receives the demangled command name, runs the command's usability check, and inverts it: returns 0 ("hide") when the command is not usable, and non-zero ("show") when it is usable (or declares no usability predicates).

Parameters:

• demangled -- [in] Demangled command name passed by _knit_command_hidden.

Returns:

0 if the command is not usable (hide it), non-zero otherwise.

int _knit_invoke_command()

Invoke a command.

Example:

_knit_invoke_command "say" "hello" "--name" "Matthieu"

Will invoke the command "say:hello" with arguments "--name" and "Matthieu".

Parameters:

• ...commands -- [in] Commands and subcommands.

• ...args -- [in] Arguments for the command.

int _knit_is_builtin()

Mark the item currently being defined as a framework builtin. This is called by knit's own source files immediately after a registration or enum definition, so that "describe" (and --exclude-builtins) can tell knit's own commands/enums apart from user-declared ones. It has two behaviors:

• Inside a command registration (KNIT_CURRENT_COMMAND set, i.e. between knit_register/knit_register_wrapper/... and knit_done), it marks the current command by setting _KNIT_CMD<cmd>_is_builtin=true.

• Otherwise it marks the most recently defined enum (_KNIT_LAST_ENUM) by adding it to the _KNIT_BUILTIN_ENUMS set.

int _knit_name_is_valid()

Checks that a parameter or command name is valid, i.e. it has to start with a letter, followed by any number of alphanumerical characters and hyphens and underscores. The names "true", "false", "null", "and", "or", and "not" are reserved for use in --when constraint expressions and are not allowed.

Parameters:

• param -- [in] Parameter name to normalize.

int _knit_name_normalize()

Normalizes a parameter or command name, i.e. converts its hyphens into underscores.

Parameters:

• name -- [in] Name to normalize.

int _knit_output_default()

This function returns the default value of an output for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the default value.

• cmd -- [in] Command to which the output belongs (must be mangled).

• output -- [in] Name of the output (must be normalized).

int _knit_output_description()

This function returns the description of an output for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the description.

• cmd -- [in] Command to which the output belongs (must be mangled).

• output -- [in] Name of the output (must be normalized).

int _knit_output_type()

This function returns the type of an output for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the output type.

• cmd -- [in] Command to which the output belongs (must be mangled).

• output -- [in] Name of the output (must be normalized).

int _knit_param_check_declaration()

This function carries out all the checks for a parameter to be declared by knit_with_required/optional/flag. The parameter name must include a type annotation in the form "name:type" (e.g. "width:integer").

Parameters:

• suffix -- [in] Suffix ("required", "optional", or "flag") to use for variables.

• param -- [in] Parameter name followed by ":type".

• description -- [in] Description of the parameter.

int _knit_param_default()

This function returns the default value of a parameter for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the default value.

• cmd -- [in] Command to which the parameter belongs (must be mangled).

• param -- [in] Name of the parameter (must be normalized).

int _knit_param_description()

This function returns the description of a parameter for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the description.

• cmd -- [in] Command to which the parameter belongs (must be mangled).

• param -- [in] Name of the parameter (must be normalized).

int _knit_param_type()

This function returns the type of a parameter for a given command.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the type.

• cmd -- [in] Command to which the parameter belongs (must be mangled).

• param -- [in] Name of the parameter (must be normalized).

int _knit_print_command_usage()

Print the help message for a command/subcommand.

Parameters:

• ...cmds -- [in] Command and subcommand names

int _knit_print_options_block()

Print the "Options"-style listing for a single command: a titled section with an hrule, then (optionally) the "--help" entry, then the command's required parameters, optional parameters, and flags, each column-aligned. Extracted so it can be printed both for a command's own options and, for a subcommand invoked through a dispatcher, for the dispatcher's options as well.

Parameters:

• cmd -- [in] Mangled command name whose options to print.

• title -- [in] Section title (e.g. "Options" or "submit options").

• with_help -- [in] "true" to include the "--help" entry, "false" otherwise.

int _knit_provenance_enabled()

Decide whether an invocation of a command participates in the provenance graph: a participating command records a "call" edge and acts as an in-process source frame for the commands it invokes; a non-participating one is transparent (no edge, and skipped when a callee resolves its edge source — see _knit_resolve_source_context).

The effective setting is resolved in this order (innermost/most-specific wins):

1. the command's own explicit mark (knit_with_provenance -> "with", knit_without_provenance -> "without"), if any;

2. otherwise the nearest marked lexical ancestor's mark, walking the colon-nested command name up via _knit_command_get_parents (e.g. "a:b:c" -> "a:b" -> "a"), so one mark on a parent governs a whole subtree;

3. otherwise the default by visibility: hidden commands (marked via knit_hidden, e.g. the "_run" worker) are transparent so internal plumbing never shows up in the graph; every other (visible) command participates.

Inheritance is over the lexical command hierarchy (the names), not the runtime call stack. Data-row recording (knit_with_table) is orthogonal to this.

Returns 0 (success) when the command participates, 1 otherwise.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_pset_filter_build()

Build the set of normalized parameter names named by a filter list, failing if any name is not in the source parameter set. Prevents a typo from silently importing the whole set. Names are normalized (hyphens and underscores are interchangeable).

Parameters:

• __knit_ret -- [out] Associative array to populate with the normalized names.

• pset_ns -- [in] Namespace prefix of the source set (e.g. "_KNIT_PSET_foo").

• set_name -- [in] Original set name, for error messages.

• mode -- [in] Filter mode ("exclude" or "only"), for error messages.

• list -- [in] Comma-separated list of parameter names.

int _knit_pset_import_skip()

Decide whether a parameter should be skipped while importing a parameter set, given the active filter. Returns 0 (skip) when the mode is "exclude" and the parameter is in the filter, or when the mode is "only" and the parameter is NOT in the filter. Returns 1 (import) otherwise, including when no filter is active.

Parameters:

• mode -- [in] Filter mode ("exclude", "only", or "" for no filter).

• filter_name -- [in] Associative array holding the filter's normalized names.

• param -- [in] Normalized parameter name under consideration.

Returns:

0 to skip the parameter, 1 to import it.

int _knit_push_done_cb()

In the context of a knit_register, push a callback to be called at the next call to knit_done. Multiple callbacks may be pushed; they are all called in reverse order of installation. The callback list is cleared after knit_done.

Parameters:

• ... -- [in] Callback function and its arguments.

int _knit_record_invocation()

Record the just-completed invocation of a command: its data row (when the command declared a table with knit_with_table) and, when the command participates in the provenance graph (see _knit_provenance_enabled), a "call" edge from its source (the caller). Both are written only when the experiment is bootstrapped.

The row id is the id resolved when this frame was pushed (see _knit_resolve_row_id) and read back from the top of _KNIT_EXECUTING_ROW_ID, so it matches the id a nested callee already saw as its source. Recording therefore runs while the frame is still on the stacks (before it is popped).

The four cases:

• transparent (hidden) command with no table: nothing to record;

• transparent command with a table: the data row only (excluded from the graph — provenance participation and data-row recording are orthogonal);

• participating command with a table: the data row and the "call" edge in one transaction (_knit_db_record_invocation);

• participating command with no table: the "call" edge on its own (_knit_prov_record_edge), whose target id joins to no data row (dangling).

The edge's start_time was captured when the frame was pushed (top of _KNIT_EXECUTING_START_TIME); its end_time is captured here, after the body and after-callbacks.

Parameters:

• cmd -- [in] Mangled command name.

• ... -- [in] The expanded invocation arguments.

int _knit_record_row_now()

Record the current invocation's database row immediately, instead of waiting for the automatic post-invocation recording. Use this when a command must persist its row before doing blocking work whose side effects update that same row — e.g. knit submit records the submission before a --wait dispatch, so the job's own state transitions (running/completed) land on an existing row. Recording is idempotent: the automatic recording afterwards sees this one and does not insert a duplicate. Must be called from within an executing command.

Parameters:

• ... -- [in] The invocation arguments (params/flags to record).

int _knit_register_checksum()

Wire up existence checking and content checksums for a "file"/"directory" parameter or output that was just declared for the command being registered.

For every checksummable declaration (whether or not --no-checksum was given) it records a per-parameter marker — a KNIT_CMD<cmd>fileparams set plus _KNIT_CMD<cmd>fileparam holding "<direction>:<kind>:<checksum>", e.g. "input:file:yes" or "output:directory:no" — that the runtime reads to enforce direction-aware existence and to know whether and how to hash. Existence is a property of the type, so it is enforced even under --no-checksum; only the digest is opted out.

Unless --no-checksum was given it additionally registers an implicit recorded output column "<name>-checksum" (DB "<name>_checksum") of type string, so the digest is written by the normal row-recording machinery and appears in the command's table.

Passing --no-checksum on a non-checksummable type is a fatal declaration error (there is nothing to checksum). A synthesized companion name that collides with an already-declared parameter, flag, or output is a fatal declaration error so the companion can never overwrite a user-declared column.

Only meaningful in a command context; wiring is skipped in a parameter set (which has no outputs and no runtime existence hooks). The parameter must already have been added to its declaration set before this is called, so the collision check sees it.

Parameters:

• direction -- [in] "input" (parameter) or "output".

• type -- [in] The declared type (name or alias) of the parameter/output.

• name -- [in] The declared (un-normalized) parameter/output name.

• no_checksum -- [in] "true" if --no-checksum was given, "false" otherwise.

int _knit_register_fileparam()

Record the per-parameter existence/checksum marker for a "file"/"directory" declaration of the command being registered: a KNIT_CMD<cmd>fileparams set plus _KNIT_CMD<cmd>fileparam holding "<direction>:<kind>:<checksum>" (e.g. "input:file:yes" or "output:directory:no"). The runtime reads it to enforce direction-aware existence and to know whether and how to hash. The type alias is resolved to its canonical kind before storing.

This is the marker half of a file/directory declaration. _knit_register_checksum calls it and then also adds a companion "<name>-checksum" column; knit_with_output_artifact calls it on its own, because an artifact records its digest in the artifacts table rather than in a column of the command's own table.

Parameters:

• direction -- [in] "input" (parameter) or "output".

• type -- [in] The declared type (name or alias).

• name -- [in] The declared (un-normalized) parameter/output name.

• checksum -- [in] "yes" to also hash at runtime, "no" to only check existence.

int _knit_register_result()

Mark an output of the command being registered as a result: something that constitutes what the experiment was for, as opposed to an incidental output. The mark is importance only; it carries no runtime behavior beyond how the command is described. It is valid on an output of any type.

The output is added to a per-command KNIT_CMD<cmd>_results set, created as associative on first use (knit_register does not create it, since not every command has a result).

Only meaningful in a command context; a call in a parameter set is a no-op. The output must already have been added to the outputs set before this is called.

Parameters:

• name -- [in] The declared (un-normalized) output name.

int _knit_reserve_name()

Reserve a declared name in a command's (or parameter set's) name space, failing if the name is already taken. Parameters, outputs, artifacts, and synthesized checksum columns all share one name space per command: a parameter and an output map to the same table column, an artifact is referred to by name at runtime, and every declared name must be unambiguous. This is the single place that enforces that, so a new kind of declaration only has to call it rather than test every other declaration set by hand.

The name space lives in the associative array <ns>_names, mapping a normalized name to the lowercased kind that claimed it ("parameter", "output", "artifact", or "checksum column"). The map is created together with the command / parameter set, so it always exists here. On a clash the message distinguishes a same-kind redeclaration ("already declared") from a cross-kind collision ("collides with a

declared <kind>").

Parameters:

• ns -- [in] Namespace prefix ("_KNIT_CMD_<cmd>" or "_KNIT_PSET_<name>").

• context_name -- [in] Human-readable command / parameter set name for messages.

• kind -- [in] Capitalized kind of the incoming name ("Parameter", "Output", "Artifact", or "Checksum column").

• display -- [in] Un-normalized name as written, for messages.

• normalized -- [in] Normalized name to reserve.

int _knit_resolve_default()

Resolve a declared default value into the value actually used when an optional parameter is not provided. A default written as "ENV[NAME]" means "fall back to

the value of the NAME environment variable"; it resolves to that variable's current value (the empty string when the variable is unset). Any other string, including one that merely looks like ENV[...] but does not name a valid shell variable, is returned unchanged, so ordinary defaults keep their literal value.

Parameters:

• raw -- [in] Raw default value as declared with knit_with_optional.

int _knit_resolve_row_id()

Resolve the row id for an invocation of a command, used when its frame is pushed onto KNIT_EXECUTING_ROW_ID. The precedence is: an explicit id already set via _knit_set_row_id (_KNIT_CMD<cmd>_row_id), otherwise a fresh uuidv7.

Every invocation gets its own distinct id: the historical couplings that made a job body reuse its submission's UUID (KNIT_JOB_PREFIX) and an app's rank-0 row reuse its run's UUID (KNIT_RUN_ID) are gone — the provenance edge, not a shared id, links a child back to its parent.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_resolve_source_context()

Resolve the source of the currently-recording invocation's "call" edge (the caller), writing the source's row id and demangled command name into the two named output variables. The recording invocation is the edge's target. The precedence (innermost wins) is:

1. In-process caller frame — the nearest participating frame below the top of _KNIT_EXECUTING_COMMAND (transparent frames, per _knit_provenance_enabled, are skipped). Covers in-process nesting and setup dispatch (boundaries B1/B4). Its id comes from the parallel _KNIT_EXECUTING_ROW_ID stack.

2. Exported env — KNIT_SOURCE_ID / KNIT_SOURCE_COMMAND, set by a caller across a process boundary (a job's batch script, a run's launcher subshell; boundaries B2/B3, wired in later milestones) and read by the first command in the re-entered process, which has no in-process caller.

3. Root — neither available: both outputs are empty.

The current frame is the top of the stacks (it is recorded before being popped), so the in-process search starts one below the top.

Parameters:

• out_id -- [out] Name of the variable to receive the source row id.

• out_name -- [out] Name of the variable to receive the source command name.

int _knit_run_after()

In the context of a knit_register, install a callback to run after the command currently being registered.

Example:

knit_register ...
knit_run_after echo "Running after command"

int _knit_run_before()

In the context of a knit_register, install a callback to run before the command currently being registered.

Example:

knit_register ...
knit_run_before echo "Running before command"

int _knit_set_row_id()

Set the "id" value of the row that will be recorded for the currently executing command (see M10 run recording). Use this when a command already owns a canonical identifier — e.g. knit submit records the job UUID it generated — instead of letting recording mint a fresh uuid. Must be called from within an executing command function.

Parameters:

• id -- [in] The uuid to record as the row's id.

int _knit_usable_before_bootstrap_validate()

knit_done callback registered by knit_usable_before_bootstrap. Enforces the three rules a command usable before bootstrap must satisfy. Each rule guards a behavior that would otherwise degrade silently (not crash) before bootstrap, so the point is to guarantee correct pre-bootstrap behavior, not to avoid a crash:

1. No database table: table row recording is skipped before bootstrap, so a usable command declaring a table would silently record nothing.

2. No "--when" constraint on any parameter: constraint evaluation (via jq) is skipped before bootstrap, so a usable command's constraint would be silently ignored.

3. Parent must also be usable before bootstrap: keeps the usable set a connected subtree rooted at the top level, which is what makes the "--help" filtering and runtime guard correct without extra reachability logic.

Any violation is fatal, naming the command and the specific reason.

Parameters:

• cmd -- [in] Command (mangled name) being validated.

int _knit_wrapper_reject_declaration()

Fatal if the command currently being registered is a wrapper. Called by the declaration functions that a wrapper is not allowed to use (parameters, outputs, dispatch, parameter sets). A parameter-set definition is never a wrapper, so the check is skipped when no command is being registered.

Parameters:

• directive -- [in] Name of the calling directive (for the error message).

Variables

String _KNIT_CALL_ALIAS

One-shot call-site alias for the next command invocation. knit_as sets it (to the user-supplied alias) immediately before delegating to knit; the first _knit_invoke_command it reaches captures it into _KNIT_EXECUTING_ALIAS and clears it, so exactly one edge — the directly named call — carries the alias.

AssociativeArray _KNIT_COMMANDS

List of registered commands.

Array _KNIT_EXECUTING_ALIAS

Stack of call-site aliases, parallel to _KNIT_EXECUTING_COMMAND: entry i holds the alias knit_as named frame i's invocation with, or empty for a plain call. Captured when the frame is pushed (from _KNIT_CALL_ALIAS, which knit_as sets just before delegating) and read back by _knit_record_invocation, which writes it to the frame's "call" edge alias column. Per-frame, so an alias on a dispatcher call never leaks onto the nested edges its body records.

Array _KNIT_EXECUTING_COMMAND

Stack of currently-executing command names (mangled). Used by knit_output.

Array _KNIT_EXECUTING_ROW_ID

Stack of resolved row ids, parallel to _KNIT_EXECUTING_COMMAND: entry i holds the id that frame i's invocation will record. The id is resolved when the frame is pushed (so a nested callee can read its caller's id while the caller's body still runs) and read back by _knit_record_invocation, so the id a callee saw as its edge source is exactly the id the caller records. _knit_set_row_id updates the top entry so an explicit id and the recorded id never diverge.

Array _KNIT_EXECUTING_START_TIME

Stack of start timestamps, parallel to _KNIT_EXECUTING_COMMAND: entry i holds the epoch seconds captured when frame i was pushed (before its body ran). _knit_record_invocation reads the top entry as the call edge's start_time and pairs it with an end_time captured at record time (after the body and after-callbacks), so an invocation's duration is a plain subtraction.

String _KNIT_INVOCATION_END_TIME

Completion timestamp of the current invocation, captured after its body and after-callbacks but before any output checksum is computed, so hashing a (possibly large) output is excluded from the recorded duration. When non-empty _knit_record_invocation uses it as the call edge's end_time instead of reading the clock itself; it is read once and cleared, so the eager and wrapper record paths (which never set it) keep capturing the clock at record time.

Array _KNIT_INVOCATION_RAW_ARGS

Raw (pre-expansion) arguments of the current command invocation. Set by _knit_invoke_command from the exact tokens the user typed, before optional defaults and flag values are spliced in, so a command body can tell an option the user typed apart from one that took its default. Read it with _knit_arg_was_provided.

The array is overwritten at the start of every invocation, including nested ones. A command body that runs nested commands (for example "bootstrap", which calls "knit setup" and "metadata store") MUST copy this global into a local at the very top of its body, before any nested call, or the nested call overwrites it. This is the copy-immediately contract.

String _KNIT_LAST_ROW_ID

Row id of the most recently recorded invocation. Exposed so a dispatcher can learn the id resolved for a body it invoked, after that body returns and its entry has been popped off _KNIT_EXECUTING_ROW_ID. knit setup uses it to write the setup body's row id to .setup.id, so a later consumer can record a "used_by" edge to the setup by id rather than by matching directory paths.

AssociativeArray _KNIT_PARAMETER_SETS

Set of defined parameter set names (normalized).

String _KNIT_RECORDING_SUPPRESSED

When non-empty, output recording is suppressed: knit_output discards its value (with a warning) and _knit_record_invocation records no row. This is a generic recording concept (the CLI layer stays unaware of MPI); the knit run per-rank worker sets it on every rank but rank 0, so a run's outputs and per-app row are recorded exactly once even though every rank re-enters the app command.

Array _KNIT_ROOT_COMMANDS

Top-level commands (those with no parent), in registration order. Together with the per-command "_KNIT_CMD_<cmd>_subcommands" arrays this forms an explicit command-tree adjacency so help/describe can traverse the tree in declaration order without deriving it from prefix-matching _KNIT_COMMANDS.

AssociativeArray _KNIT_USED_ALIASES

Set of call-site aliases already used within each invocation, so knit_as can reject a reused alias (which would make two edges indistinguishable in a query). Keyed by "<parent-row-id>:<alias>", where the parent row id scopes the alias to the calling invocation (empty for a root-level call).

db.sh

Functions

int _knit_db_check_table()

Check whether a table exists in the Knit database and matches the given column specifications exactly (count, names, types, and order). Returns 0 if the table exists and matches, 1 if the table does not exist, or 2 if the table exists but the schema differs from what was specified.

Example:

_knit_db_check_table "runs" "id:uuid" "duration:real"
# returns 0, 1, or 2

Parameters:

• table_name -- [in] Name of the table to check.

• ...specs -- [in] One or more "column-name:type" specifications.

Returns:

0 if the table matches, 1 if absent, 2 if schema differs.

int _knit_db_create_table()

Create a new table in the Knit database. Each column specification must be of the form "name:type" where type is a valid Knit type. Column names are normalized (hyphens converted to underscores). Fails with a fatal error if the table already exists, if no columns are specified, if a column spec is malformed, or if a type is unknown.

Example:

_knit_db_create_table "runs" "id:uuid" "duration:real" "label:string"

Parameters:

• table_name -- [in] Name of the table to create.

• ...specs -- [in] One or more "column-name:type" specifications.

int _knit_db_migrate_table()

Migrate an existing table to a new column schema. Each column specification may be "name:type" (for columns that already exist or are being retyped) or "name:type=default" (required for columns not present in the current schema, so that existing rows can be back-filled with the given default value). Columns absent from the new spec are dropped. Column names are normalized (hyphens converted to underscores). If the current schema already matches the desired schema the function returns 0 without touching the database.

The default value is always treated as a SQL string literal; SQLite's type affinity coercion handles integer/real columns correctly.

Example:

_knit_db_migrate_table "runs" "id:uuid" "count:integer=0" "label:string"

Parameters:

• table_name -- [in] Name of the table to migrate.

• ...specs -- [in] One or more "name:type" or "name:type=default" specs.

Returns:

0 if the migration was applied or no migration was needed.

int _knit_db_record_invocation()

Insert one row into a command's table, recording an invocation, and — when a provenance edge is requested — the matching edge into the provenance table, both in a single transaction. The row is built from the command's declared schema: the "id" column (a caller-supplied uuid), then the value of every required parameter, optional parameter, and flag (read from the expanded invocation arguments), then every output (read from the in-memory KNIT_CMD<cmd>_output_value store populated by knit_output, falling back to the output's declared default). Column names are the normalized (underscored) knit names, matching the schema created by _knit_db_setup_table.

The provenance edge, when requested, has the recorded row as its target (target_id = id, target_name = the demangled command name). An empty edge_type means "record no edge" — the row is inserted on its own, unchanged from before provenance existed. A non-empty edge_type (e.g. "call") writes both the row and the edge atomically, so a partial state cannot be observed.

Parameters:

• cmd -- [in] Mangled command name (as used in KNIT_CMD* variables).

• table -- [in] Table to insert into.

• id -- [in] Value for the "id" column (the target's uuid).

• source_id -- [in] Provenance edge source id (empty for a root); see prov.sh.

• source_name -- [in] Provenance edge source name (empty for a root).

• edge_type -- [in] Edge type (e.g. "call"), or empty to record no edge.

• start_time -- [in] Edge start_time (epoch seconds, empty -> NULL).

• end_time -- [in] Edge end_time (epoch seconds, empty -> NULL).

• alias -- [in] Edge call-site alias (empty -> NULL); see prov.sh.

• ... -- [in] The expanded invocation arguments (params/flags to read).

int _knit_db_setup_table()

Done callback installed by knit_with_table. Inspects the registered parameters, flags, and outputs of the command and ensures the database table matches that schema — creating it if absent or migrating it if the schema has changed.

Column order: "id" (uuid) first, then required parameters, optional parameters, flags, and outputs, each group sorted alphabetically.

For migration defaults:

• Optional parameters use their declared default value.

• Outputs use their declared default value.

• Required parameters and flags use a type-based default (0, false, or "").

Parameters:

• cmd -- [in] Mangled command name (as used in KNIT_CMD* variables).

• table_name -- [in] Name of the database table to create or migrate.

int _knit_db_sql_ident()

Wrap an SQL identifier (table or column name) in double quotes, escaping any embedded double-quote characters by doubling them, per the SQL standard.

Example:

local q; _knit_db_sql_ident q "my_table"   # q == "my_table"
local q; _knit_db_sql_ident q 'a"b'        # q == "a""b"

Parameters:

• __knit_ret -- [out] Name of the variable to hold the quoted identifier.

• name -- [in] Identifier to quote.

int _knit_db_type_default()

Return a sensible default value string for a given Knit type. Used when migrating a table to provide a back-fill value for newly added columns that do not have a user-supplied default.

Example:

local d; _knit_db_type_default d "integer"  # d == 0
local d; _knit_db_type_default d "boolean"  # d == false
local d; _knit_db_type_default d "string"   # d == (empty)

Parameters:

• __knit_ret -- [out] Name of the variable to hold the default value.

• type -- [in] Knit type name or alias.

int _knit_db_update_row()

Update columns of an existing row, identified by its "id". Each assignment is a "column=value" string; the column is a knit name (normalized to underscores to match the schema). Used to record later state transitions of a recorded invocation (e.g. a job moving to "completed").

Parameters:

• table -- [in] Table to update.

• id -- [in] Value of the "id" column identifying the row.

• ... -- [in] One or more "column=value" assignments.

Variables

AssociativeArray _KNIT_DB_REGISTERED_TABLES

Associative array mapping table name to the demangled command name that registered it. Used to detect duplicate table use across commands.

describe.sh

Functions

int _knit_describe()

Body of the "describe" command: read the requested filters and --format, then emit the description in that format. With "--output <file>" the document is written to that file instead of standard output (which also disables the default format's auto-color, since the destination is not a terminal).

Parameters:

• ... -- [in] Command arguments (expanded by the CLI framework).

int _knit_describe_children()

Return the mangled names of the direct children of a command as an array, in registration (declaration) order. The parent is given as a mangled name, or the empty string to list the top-level (root) commands. This reads the command tree adjacency built at registration time (KNIT_ROOT_COMMANDS and the per-command "_KNIT_CMD</em><cmd>_subcommands" arrays), so it is fork-free and needs no per-invocation build/teardown.

int _knit_describe_command_kind()

Print the structural kind of a command, read from the _type field set at registration: "wrapper" (knit_register_wrapper), "job" (knit_register_job), "app" (knit_register_app), "setup" (a setup command), or "command" otherwise. This is the registered kind, not a guess from the parent: "submit" has both jobs and ordinary subcommands (submit prepared / submit next), so a child of a dispatcher is not necessarily a job.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the command kind.

• cmd -- [in] Mangled command name.

int _knit_describe_default()

Emit the human-readable description of the experiment: one titled block per command, walked depth-first so a subcommand follows its parent. Color is enabled only when stdout is a terminal and "--no-color" is not given.

Parameters:

• ... -- [in] Command arguments (expanded by the CLI framework).

int _knit_describe_default_artifacts()

Print a command's "Artifacts" section for the human-readable format (name, then "[type] description", with a trailing ", result" in the bracket for an artifact that carries "--result"). An artifact has no default, so none is shown. The artifacts are read from the command's artifacts set; the caller prints this section only for a command that declares one.

Parameters:

• cmd -- [in] Mangled command name.

• use_color -- [in] "true" to emit ANSI styling in the header.

• indent -- [in] Leading indentation for the section header (defaults to none); entries are indented two further spaces.

int _knit_describe_default_command()

Print one command as a titled block for the human-readable format, then recurse (flat, depth-first) into its emitted subcommands so each command is its own block titled by its full space-separated name. The Options and Outputs sections honor the "--no-input-params" / "--no-output-params" filters, an "Extra" section is printed when the command declares post-"--" arguments, and an "Implementation" section (the function body) is printed for a user command when "--include-implementation" is set.

Parameters:

• cmd -- [in] Mangled command name.

• use_color -- [in] "true" to emit ANSI styling.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection.

int _knit_describe_default_heading()

Print a title or section header for the human-readable format. With color it is rendered bold and underlined on its own line; without color it is followed by an "---" hrule of matching width (the style "--help" uses).

Parameters:

• text -- [in] Header text.

• use_color -- [in] "true" to emit ANSI styling, "false" for an hrule.

• indent -- [in] Leading indentation (defaults to none).

int _knit_describe_default_options()

Print a command's "Options" section for the human-readable format, mirroring the "--help" layout: a header, the "--help" entry, then required, optional (with "default: '…'") and flag parameters, column-aligned, each annotated with its enum constraint, "resource:" type (for a knit_with_resource parameter), "artifact:" kind (for a knit_with_input_artifact parameter), and "when:" clause when present.

Parameters:

• cmd -- [in] Mangled command name.

• use_color -- [in] "true" to emit ANSI styling in the header.

• indent -- [in] Leading indentation for the section header (defaults to none); entries are indented two further spaces.

int _knit_describe_default_outputs()

Print a command's "Outputs" section for the human-readable format (name, then "[type, default: '…'] description", with a trailing ", result" in the bracket for an output that carries "--result"). An artifact is not an output column, so it is not listed here; it gets its own "Artifacts" section.

Parameters:

• cmd -- [in] Mangled command name.

• use_color -- [in] "true" to emit ANSI styling in the header.

• indent -- [in] Leading indentation for the section header (defaults to none); entries are indented two further spaces.

int _knit_describe_emit()

Emit the description in the requested format to standard output. Split out from _knit_describe so the caller can redirect the whole document to a file for "--output" without duplicating the format dispatch. "--compact" selects the single-line JSON variant and applies only to the "json" format.

Parameters:

• format -- [in] Output format ("default", "json", "yaml", or "markdown").

• compact -- [in] "true" to emit compact single-line JSON (json format only).

• ... -- [in] Command arguments (expanded by the CLI framework).

int _knit_describe_emit_array()

Emit a JSON array from a list of already-rendered elements. The opening bracket is printed inline; each element must already carry its own indentation; the closing bracket is printed at the given indent. An empty element list yields "[]".

Parameters:

• indent -- [in] Indentation string for the closing bracket.

• ...elements -- [in] Pre-rendered array elements (indented).

int _knit_describe_emit_object()

Emit a JSON object from a list of already-rendered entries. The opening brace is printed inline (the caller positions it, e.g. right after a "key": prefix); each entry must already carry its own indentation; the closing brace is printed at the given indent. An empty entry list yields "{}".

Parameters:

• indent -- [in] Indentation string for the closing brace.

• ...entries -- [in] Pre-rendered "key": value fragments (indented).

int _knit_describe_enum_constraint()

Return a "one of: a, b, c" constraint string for an enum-typed parameter, or the empty string when the parameter's type is not an enum. Values are sorted to match the other formatters.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the constraint string.

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter name.

int _knit_describe_enum_values_json()

Return the values of an enum as an inline JSON array of strings (sorted for a stable order).

Parameters:

• __knit_ret -- [out] Name of the variable to hold the JSON array.

• name -- [in] Enum type name.

int _knit_describe_filter_on()

Return success if the named boolean filter is enabled for the current invocation. Unknown or unset keys are treated as disabled.

Parameters:

• name -- [in] Filter key (see _KNIT_DESCRIBE_FILTERS).

int _knit_describe_implementation()

Print a command's implementation — its registered function body as produced by "declare -f" — when "--include-implementation" is active and the command is a user (non-builtin) command. Prints nothing otherwise: without the flag no body is emitted, and a builtin's body is knit implementation detail that is never dumped. Every formatter consults this so the rule is applied in one place.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_describe_is_result()

Return success if an output of a command carries the "--result" mark (it is a member of the command's results set). The set does not exist for a command with no result, in which case the membership test simply fails.

Parameters:

• cmd -- [in] Mangled command name.

• output -- [in] Normalized output name.

int _knit_describe_is_selected()

Decide whether a command's own block should be rendered, as opposed to the command being kept only as a path container for a selected descendant. Returns success when:

• no "--only" selection is active (every command is rendered), or

• the command is itself selected by "--only", or

• an ancestor is selected and "--recursive" is set. A command that is emitted only to preserve the path down to a selected descendant returns failure. The flat and Markdown formatters use this to skip a container's block (the child's full name already conveys the hierarchy) while still recursing into it; the tree formats (JSON/YAML) ignore it and emit every command _knit_describe_should_emit keeps, because a child cannot nest without its parent. Call only for a command already known to be emitted.

Parameters:

• cmd -- [in] Mangled command name.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection, "false" otherwise.

int _knit_describe_json()

Emit the complete description of the experiment as a JSON document: knit version, experiment (script) name, format version, the full command tree (top-level commands with nested subcommands, hidden commands excluded), and the map of user-defined enums.

int _knit_describe_json_artifact()

Render one artifact as a JSON object (array element: leading indent included). An artifact is a produced entity, not an output column, so it carries no "default"; the "result" boolean says it carries the "--result" mark.

Parameters:

• cmd -- [in] Mangled command name.

• output -- [in] Normalized artifact name.

• indent -- [in] Indentation of the object's opening brace.

int _knit_describe_json_artifacts()

Render a command's "artifacts" array (one object per declared artifact). Printed inline (object value: no leading indent). Yields "[]" for a command that declares no artifact (the set is absent then).

Parameters:

• cmd -- [in] Mangled command name.

• indent -- [in] Indentation of the array's opening bracket.

int _knit_describe_json_command()

Render a single command (and, recursively, its subcommands) as a JSON object. The object is an array element, so its leading indent is included. Subcommands are pruned by the active filters (hidden/builtin/"--only"), matching the top-level command list.

Parameters:

• cmd -- [in] Mangled command name.

• indent -- [in] Indentation of the object's opening brace.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection.

int _knit_describe_json_compact()

Emit the JSON description as a single compact line (no indentation, no inter-entry newlines, and no spaces after ":" or ","). It shadows the JSON builders' whitespace separators (_KNIT_DESCRIBE_JSON_NL / _CS / _IND) with the empty string for the duration of one _knit_describe_json call, so the compact form is produced directly by the same builder tree — no second minify pass. The locals are visible to the builders (and their command-substitution subshells) through bash dynamic scoping, and are restored automatically on return.

int _knit_describe_json_enums()

Render the top-level "enums" object: every user-defined (non-builtin) enum mapped to its values. Printed inline (object value: no leading indent).

Parameters:

• indent -- [in] Indentation of the object's opening brace.

int _knit_describe_json_escape()

Escape a string so it can be embedded inside a JSON string literal, without any external dependency (no jq), so "describe" works on a fresh checkout before bootstrap. Backslashes and double quotes are backslash-escaped; newlines, CR, and tabs become their short escapes; any remaining control character becomes a "\uXXXX" escape. The surrounding quotes are NOT added (see _knit_describe_json_str).

Parameters:

• __knit_ret -- [out] Name of the variable to hold the escaped string.

• string -- [in] String to escape.

int _knit_describe_json_output()

Render one output as a JSON object (array element: leading indent included). The "result" boolean says the output carries the "--result" mark. An artifact is not an output column, so it is not rendered here (see _knit_describe_json_artifact).

Parameters:

• cmd -- [in] Mangled command name.

• output -- [in] Normalized output name.

• indent -- [in] Indentation of the object's opening brace.

int _knit_describe_json_outputs()

Render a command's "outputs" array (value output columns only; artifacts get their own "artifacts" array). Printed inline (object value: no leading indent).

Parameters:

• cmd -- [in] Mangled command name.

• indent -- [in] Indentation of the array's opening bracket.

int _knit_describe_json_param()

Render one input parameter as a JSON object (array element: the leading indent is included). The rendered fields depend on the group: "required" has no default, "optional" adds a raw default, and "flags" are always boolean with no default or enum. An enum-typed parameter inlines its allowed values, a resource parameter (knit_with_resource) adds its "resource" type, an input-artifact parameter (knit_with_input_artifact) adds its required "artifact" kind, and a "--when" constraint is included when present.

Parameters:

• cmd -- [in] Mangled command name.

• group -- [in] Parameter group ("required", "optional", or "flags").

• param -- [in] Normalized parameter name.

• indent -- [in] Indentation of the object's opening brace.

int _knit_describe_json_params()

Render a command's "parameters" object (required, optional, flags arrays, and the extra description). Printed inline (object value: no leading indent).

Parameters:

• cmd -- [in] Mangled command name.

• indent -- [in] Indentation of the object's opening brace.

int _knit_describe_json_str()

Return a value as a quoted, escaped JSON string literal.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the JSON string literal.

• string -- [in] Value to render.

int _knit_describe_markdown()

Emit the complete description of the experiment as a single Markdown document: a "#" title (the program description, or the script name when unset), a "##

Commands" wrapper, and one flat "###" section per command (depth-first, so a subcommand follows its parent). Depth does not consume heading levels, keeping the scheme within Markdown's six-level limit. Enum values are inlined in each parameter's Constraints column rather than a separate section.

int _knit_describe_md_artifacts()

Print a command's "#### Artifacts" sub-section as a Markdown table (one row per artifact). A "Result" column shows "result" for an artifact that carries "--result". An artifact has no default, so no Default column is shown. The artifacts are read from the command's artifacts set; the caller prints this sub-section only for a command that declares one.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_describe_md_cell()

Escape a value so it is safe inside a Markdown table cell: pipes are backslash- escaped (they otherwise start a new column) and newlines/carriage returns are folded to spaces (a table row must stay on one line).

Parameters:

• __knit_ret -- [out] Name of the variable to hold the escaped value.

• value -- [in] Value to escape.

int _knit_describe_md_code()

Render a value as a Markdown inline-code span (escaped for a table cell). An empty value yields nothing, so this doubles as the "Default" column renderer: a declared value is shown as code and an empty-string default leaves a blank cell.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the rendered code span.

• value -- [in] Value to render.

int _knit_describe_md_command()

Render one command as a "### <full name>" Markdown section (an intro line with its kind/builtin note and description, an italic "Extra" line when declared, the "#### Parameters" / "#### Outputs" sub-sections honoring the omit flags, and a fenced "#### Implementation" block for a user command when "--include-implementation" is set), then recurse (flat, depth-first) into its emitted subcommands so each command is its own "###" section regardless of depth.

Parameters:

• cmd -- [in] Mangled command name.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection.

int _knit_describe_md_constraints()

Build the "Constraints" column text for a parameter: the enum "one of: …" list (when the type is an enum), the "resource: <type>" annotation (when the parameter was declared with knit_with_resource), the "artifact: <kind>" annotation (when the parameter was declared with knit_with_input_artifact), and the "--when" clause (with the raw expression in inline code), joined by "; ". Returns the empty string when the parameter is unconstrained.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the constraints text.

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter name.

int _knit_describe_md_outputs()

Print a command's "#### Outputs" sub-section as a Markdown table (one row per output). A "Result" column shows "result" for an output that carries "--result". An artifact is not an output column, so it is not listed here; it gets its own "#### Artifacts" sub-section. Prints "*None.*" when the command declares no output.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_describe_md_params()

Print a command's "#### Parameters" sub-section as a Markdown table (one row per required, optional, and flag parameter, in that order; each group sorted), with a "Kind" column marking the group. Prints "*None.*" when the command declares no parameters. The universal "--help" flag is intentionally omitted.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_describe_read_filters()

Populate the module-level filter state (_KNIT_DESCRIBE_FILTERS and _KNIT_DESCRIBE_ONLY) from the current invocation's arguments, so the model/traversal layer applies the requested filtering. Called by _knit_describe before any formatter runs.

Parameters:

• ... -- [in] Command arguments (expanded by the CLI framework).

int _knit_describe_should_emit()

Decide whether a command appears in the filtered command tree. A command is emitted when it is visible (passes the hidden/builtin filters) and either:

• no "--only" selection is active (every visible command is selected), or

• it is itself selected by "--only", or

• an ancestor is selected and "--recursive" is set, or

• it has a descendant that is itself emitted (so it is kept as a container that preserves the path down to a selected command).

Parameters:

• cmd -- [in] Mangled command name.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection, "false" otherwise.

int _knit_describe_visible()

Return success if a command passes the hidden/builtin filters, independent of any "--only" selection: hidden commands are dropped unless "--include-hidden" is set, and builtin commands are dropped when "--exclude-builtins" is set.

Parameters:

• cmd -- [in] Mangled command name.

int _knit_describe_yaml()

Emit the complete description of the experiment as a YAML document, serializing the identical model as _knit_describe_json (same keys, nesting, and field semantics): knit version, experiment (script) name, format version, the filtered command tree, and the map of user-defined enums.

int _knit_describe_yaml_artifact()

Render one artifact as a YAML block-sequence element (the first key carries the "- " indicator at item_indent). An artifact is a produced entity, not an output column, so it carries no "default"; the "result" boolean says it carries the "--result" mark.

Parameters:

• cmd -- [in] Mangled command name.

• output -- [in] Normalized artifact name.

• item_indent -- [in] Indentation of the "- " sequence indicator.

int _knit_describe_yaml_command()

Render a single command (and, recursively, its subcommands) as a YAML block-sequence element (the first key carries the "- " indicator at item_indent). Subcommands are pruned by the active filters (hidden/builtin/"--only"), matching the top-level command list.

Parameters:

• cmd -- [in] Mangled command name.

• item_indent -- [in] Indentation of the "- " sequence indicator.

• sel_ancestor -- [in] "true" if an ancestor of the command is in the "--only" selection.

int _knit_describe_yaml_enums()

Render the top-level "enums:" mapping: every user-defined (non-builtin) enum mapped to its values as a flow sequence. Yields "enums: {}" when there are none.

int _knit_describe_yaml_flow_seq()

Print a YAML flow sequence ("[a, b, c]") of scalar values, each double-quoted when it would be unsafe or coerced as a plain flow scalar (the plain-scalar rules plus the flow indicators , [ ] { }). An empty list yields "[]".

Parameters:

• __knit_ret -- [out] Name of the variable to hold the flow sequence.

• ...values -- [in] Scalar values.

int _knit_describe_yaml_needs_quote()

Return success when a single-line string value must be double-quoted to survive a YAML round-trip as a string: leaving it as a plain scalar would either coerce it to another type (number, boolean, null) or be syntactically unsafe. Double-quoting an already-safe value is harmless, so the predicate errs toward quoting. Multi-line values are handled separately as block scalars.

Parameters:

• value -- [in] String value to test.

int _knit_describe_yaml_output()

Render one output as a YAML block-sequence element (the first key carries the "- " indicator at item_indent). The "result" boolean says the output carries the "--result" mark. An artifact is not an output column, so it is not rendered here (see _knit_describe_yaml_artifact).

Parameters:

• cmd -- [in] Mangled command name.

• output -- [in] Normalized output name.

• item_indent -- [in] Indentation of the "- " sequence indicator.

int _knit_describe_yaml_param()

Render one input parameter as a YAML block-sequence element (the first key carries the "- " indicator at item_indent). Mirrors the JSON parameter object: an enum type inlines its allowed values, "optional" carries a raw default, flags are boolean with no default, a resource parameter (knit_with_resource) adds its "resource" type, an input-artifact parameter (knit_with_input_artifact) adds its required "artifact" kind, and a "--when" constraint is included when present.

Parameters:

• cmd -- [in] Mangled command name.

• group -- [in] Parameter group ("required", "optional", or "flags").

• param -- [in] Normalized parameter name.

• item_indent -- [in] Indentation of the "- " sequence indicator.

int _knit_describe_yaml_params()

Render a command's "parameters:" mapping body: the required/optional/flags sequences (each "[]" when empty) and the "extra" scalar (or null), with the group keys at keys_indent. The "parameters:" key line itself is printed by the caller.

Parameters:

• cmd -- [in] Mangled command name.

• keys_indent -- [in] Indentation of the required/optional/flags keys.

int _knit_describe_yaml_scalar()

Render a string value as a YAML scalar to follow "key: ". A multi-line value becomes a literal block scalar ("|-") whose lines are indented by cont_indent; a single-line value that would be coerced or is unsafe as a plain scalar is double-quoted (reusing the JSON escaper, whose escapes YAML's double-quoted style shares); anything else is emitted verbatim.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the rendered scalar.

• value -- [in] String value to render.

• cont_indent -- [in] Indentation prepended to each line of a block scalar.

Variables

String _KNIT_DESCRIBE_EMITTED

Scratch flag the flat formatter uses to place inter-block blank lines: 0 until the first command block is printed, 1 afterwards, so a separator precedes every block except the first. Reset at the start of _knit_describe_default. It exists because a command kept only as a path container prints nothing, so the separator cannot be tied to the tree walk.

AssociativeArray _KNIT_DESCRIBE_FILTERS

Boolean filter state for the current "describe" invocation, populated by _knit_describe from the parsed flags and consulted by the model/traversal layer so every formatter inherits the same filtering. Keys: "exclude_builtins", "no_input_params", "no_output_params", "include_hidden", and "recursive"; each value is "true" or "false". A missing key defaults to "false", so the default (no filtering) also applies when the traversal is exercised directly.

String _KNIT_DESCRIBE_JSON_CS

Insignificant space the JSON builders insert after each ":" and inline ",". Defaults to a single space (pretty); shadowed with the empty string by _knit_describe_json_compact for compact output.

String _KNIT_DESCRIBE_JSON_IND

One level of indentation for the JSON builders. Defaults to two spaces (pretty); shadowed with the empty string by _knit_describe_json_compact so all indentation collapses for compact output.

String _KNIT_DESCRIBE_JSON_NL

Inter-entry newline the JSON builders insert inside objects and arrays. Its default (a newline) produces pretty-printed JSON; _knit_describe_json_compact shadows it with the empty string to emit single-line compact JSON without a second minify pass.

AssociativeArray _KNIT_DESCRIBE_ONLY

Set of mangled command names selected by "--only" for the current "describe" invocation. Empty means no selection was given, in which case every (visible) command is described.

detect.sh

Functions

int _knit_command_path()

Print the absolute path to a system executable if it exists on PATH, or nothing when it is absent. Thin wrapper around command -v that gives the bootstrap a single, easily stubbable resolution point for the system binaries (sqlite3, jq) it may symlink instead of building from source.

Parameters:

• name -- [in] Name of the executable to look up.

int _knit_detect_job_manager()

Detect which batch job manager is available in the current environment.

Outputs one of the following strings to stdout and returns 0:

• "flux" — FLUX_URI names a live Flux instance that owns this shell

• "slurm" — sbatch is present in PATH

• "pbs" — qsub is present in PATH (and sbatch is not)

• "flux" — flux is present in PATH (and no sbatch/qsub)

• "<unknown>" — none of the above

The result is cached in _KNIT_DETECTED_JOB_MANAGER so that subsequent calls within the same session return immediately without re-probing the PATH.

FLUX_URI is tested first because Flux is often deployed inside a Slurm or PBS allocation, so flux, sbatch, and qsub can all be on PATH at once. A live FLUX_URI means a Flux instance owns this shell, so Flux is the right backend; without it a Flux-under-Slurm login shell still detects as Slurm. When both sbatch and qsub are present (but no FLUX_URI), Slurm takes priority. A system flux with no live instance is the last resort before <unknown>.

int _knit_detect_launcher()

Detect which MPI launcher to use in the current environment.

Outputs one of the following strings to stdout and returns 0:

• "flux" — the detected scheduler is Flux (flux run wins regardless of any MPI-native launcher on PATH)

• "pals" — the MPI-native probe found PALS

• "openmpi" — the MPI-native probe found OpenMPI

• "mpich" — the MPI-native probe found MPICH

• "flux" — no MPI-native launcher, but flux is in PATH

• "<unknown>" — no recognised launcher found

Detection is scheduler-aware. When the detected scheduler is Flux, "flux run" is the reliable native launcher and wins even if an MPI-native launcher is on PATH: flux run bootstraps MPI through Flux's own PMI, and a bare mpirun may not reach the compute nodes inside a Flux allocation. Under any other scheduler (slurm, pbs, none, <unknown>) an MPI-native launcher on PATH wins, because a scheduler-integrated launcher such as srun needs the MPI built against that manager's PMI (which knit cannot verify), whereas mpirun/mpiexec always match their own MPI.

The MPI-native probe is delegated to _knit_detect_mpi_launcher(). Flux is the last resort when it is not the scheduler: an MPI-native launcher wins when present, so a system flux on PATH is chosen only after that probe finds nothing.

The result is cached in _KNIT_DETECTED_LAUNCHER so that subsequent calls within the same session return immediately without re-probing the PATH.

int _knit_detect_mpi()

Detect which MPI implementation is available in the current environment.

Outputs one of the following strings to stdout and returns 0:

• "openmpi" — mpirun is present and its --version output contains "Open MPI"

• "mpich" — mpirun is present and its --version output contains "HYDRA"

• "<unknown>" — mpirun is absent or its version string is not recognised

The result is cached in _KNIT_DETECTED_MPI so that subsequent calls within the same session return immediately without re-running mpirun.

int _knit_detect_mpi_launcher()

Detect which MPI-native launcher is available on PATH, ignoring the scheduler.

Outputs one of the following strings to stdout and returns 0:

• "pals" — mpiexec is present and its --help output starts with "Parallel Application Launch Service"

• "openmpi" — mpirun is present and its --version output contains "Open MPI"

• "mpich" — mpirun is present and its --version output contains "HYDRA"

• "<unknown>" — no recognised MPI-native launcher found

This probe is deliberately scheduler-unaware and never returns "flux": it reports only the MPI-native launcher that ships with an MPI on PATH. PALS is checked first because PALS provides mpiexec but not mpirun; OpenMPI and MPICH also provide mpiexec as an alias, so the first-line check distinguishes them.

It is the right probe for a setup that declares knit_provides_launcher: such a setup always installs an MPI-native launcher, so its contract must never freeze to "flux" even when the setup runs on a Flux login node.

The result is cached in _KNIT_DETECTED_MPI_LAUNCHER so that subsequent calls within the same session return immediately without re-probing the PATH.

int _knit_detect_node_ncpus()

Detect the per-node core count of the current cluster by querying the detected batch scheduler and taking the modal (most common) value across its nodes:

• slurm: sinfo -h -N -o c'(one line per node)

• pbs:pbsnodes -aresources_available.ncpus lines

• flux:flux resource list` total cores divided by total nodes

Counting is node-weighted (-N on Slurm; one entry per node on PBS), so a minority outlier such as a login node listed alongside the compute nodes is ignored in favour of the value the bulk of the nodes report.

Prints the detected core count (a positive integer) to stdout, or nothing when the query yields no usable value. Knit allocates whole nodes, so this feeds node_ncpus (cpus-per-node) on machines that have no profile. When there is no batch scheduler the count of the local machine (nproc, or getconf as a portable fallback) is used so that node_ncpus is still populated on a workstation. The result is cached in _KNIT_DETECTED_NODE_NCPUS.

The local fallback is deliberately confined to the no-scheduler case: on a machine with a scheduler the login node's core count is not representative of the compute nodes, so a failed scheduler query returns nothing rather than the (misleading) local value.

Variables

String _KNIT_DETECTED_JOB_MANAGER

Cache for _knit_detect_job_manager(). Empty means "not yet detected"; one of "flux", "slurm", "pbs", or "<unknown>" after the first successful detection call.

String _KNIT_DETECTED_LAUNCHER

Cache for _knit_detect_launcher(). Empty means "not yet detected"; one of "pals", "openmpi", "mpich", "flux", or "<unknown>" after the first successful detection call.

String _KNIT_DETECTED_MPI

Cache for _knit_detect_mpi(). Empty means "not yet detected"; one of "openmpi", "mpich", or "<unknown>" after the first successful detection call.

String _KNIT_DETECTED_MPI_LAUNCHER

Cache for _knit_detect_mpi_launcher(). Empty means "not yet detected"; one of "pals", "openmpi", "mpich", or "<unknown>" after the first successful detection call.

String _KNIT_DETECTED_NODE_NCPUS

Cache for _knit_detect_node_ncpus(). Empty means "not yet detected" (or undetectable); a positive integer once detection succeeds.

global.sh

Functions

int _knit_stdout_is_terminal()

Return success when standard output is a terminal. Factored into its own function so callers that colorize or wrap output based on the terminal (e.g. "--help" highlighting and describe) can stub it in tests to force the terminal path on or off deterministically.

int _knit_terminal_width()

Set the nameref given as the first argument to the width of the terminal (in columns), or to "0" when the width cannot be used for wrapping — i.e. when standard output is not a terminal or "stty size" yields no usable value. Callers treat a "0" result as "no wrapping". Factored into its own function (mirroring _knit_stdout_is_terminal) so tests can stub it to force a width deterministically.

The result is returned by nameref rather than printed for capture with command substitution: the terminal check (_knit_stdout_is_terminal, i.e. "[[ -t 1 ]]") must be evaluated in the caller's context, where file descriptor 1 is the real help destination. A command substitution would replace fd 1 with a pipe and so always report "not a terminal", disabling wrapping even on a real terminal.

Parameters:

• __knit_ret -- [out] Name of the variable to receive the width (or "0").

Variables

AssociativeArray _KNIT_COLORS

Associative array mapping color and style names to their ANSI escape sequences. Use these to colorize terminal output. The "reset" entry clears all active attributes.

Foreground colors: black, red, green, yellow, blue, magenta, cyan, white, bright_black, bright_red, bright_green, bright_yellow, bright_blue, bright_magenta, bright_cyan, bright_white. Background colors: bg_black, bg_red, bg_green, bg_yellow, bg_blue, bg_magenta, bg_cyan, bg_white, bg_bright_black, bg_bright_red, bg_bright_green, bg_bright_yellow, bg_bright_blue, bg_bright_magenta, bg_bright_cyan, bg_bright_white. Styles: bold, dim, italic, underline, blink, reverse, hidden, strikethrough.

String _KNIT_IS_BOOTSTRAPPING

Set to "true" when the first argument passed to the experiment script is "bootstrap", i.e. when the user is running the bootstrap command. Functions that require a bootstrapped experiment use this to distinguish between a legitimate pre-bootstrap invocation (during bootstrap itself) and an erroneous one (calling a DB-backed command before bootstrap has run).

job.sh

Functions

int _knit_command_is_job()

Test whether a command is a job, i.e. it was registered with knit_register_job. A job's row carries a "state" column written from its callbacks and signal traps (running / killed / completed), so a directive that would suppress that row can consult this to reject a job. Reads the command kind from the KNIT_CMD<cmd>_type field (see knit_register).

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command is a job, 1 otherwise.

int _knit_declare_submit_options()

Declare the scheduler options shared by the submit and prepare dispatchers, so the two surfaces cannot drift. Must be called between knit_register and knit_done while registering either dispatcher. It declares the optional submission parameters (setup, name, job-name, account, project, queue, nodes, walltime, gpus-per-node) and the free-form --group label, plus the "job" dispatch marker and the "Jobs" subcommand title.

Each dispatcher declares the rest itself: submit adds the submit-only --wait flag and owns the "jobs" table and its outputs; prepare declares neither a table nor outputs (it records under the submit identity, see _knit_prepare_build).

int _knit_job_after_cb()

After-callback installed on every submit subcommand by knit_register_job. Marks the job "completed" once its body has returned normally.

int _knit_job_before_cb()

Before-callback installed on every setup subcommand by knit_register_job. Verifies that KNIT_JOB_PREFIX is set, ensuring the job was invoked through knit submit rather than called directly, installs the pre-termination signal handler, marks the job "running", and records the allocated hostnames.

It does NOT source the setup environment: the jobscript already sources the setup's .activate.sh before it re-enters the experiment (see _knit_sched_write_jobscript), and the exports survive the exec, so the environment is already active here. Sourcing again would re-apply the composable env_append/env_prepend lines and double their entries.

int _knit_job_killed_trap()

Signal handler installed while a job runs on the compute node. Schedulers warn a job before killing it: Slurm can send a chosen signal a configurable time before the walltime limit (requested via --signal in the batch directives) and sends SIGTERM before SIGKILL; PBS likewise sends SIGTERM before SIGKILL. Flux cancels a batch job by shutting down its instance, which sends SIGHUP to the batch initial program. This handler records the job as "killed" so its row does not stay stuck at "running", then exits so the after-callback (which would mark it "completed") does not run.

int _knit_job_record_hostnames()

Record the nodes the running job was allocated into its jobs-table row. Called on the compute side, once the job is running, where the scheduler has populated the node allocation and the experiment's .knit is shared over the parallel file system (so the row inserted by knit submit on the login node can be updated in place). The stored value is the deduplicated, comma-separated host list, i.e. the output of knit_job_hostnames --separator ,. The row id is the job UUID, i.e. the basename of KNIT_JOB_PREFIX. Best-effort: like state tracking, a failure is downgraded to a warning so it never takes down the job.

int _knit_job_set_state()

Update the lifecycle state of the running job's jobs-table row. Called on the compute side, where the experiment's .knit is shared over the parallel file system, so the row inserted by knit submit on the login node can be updated in place. The row id is the job UUID, i.e. the basename of KNIT_JOB_PREFIX (the job directory). Best-effort: status tracking must never take down the job itself, so a failure is downgraded to a warning.

Parameters:

• state -- [in] New state value (e.g. running, completed, killed).

int _knit_prepare_build()

Build a submission without dispatching it: the shared first phase of submit and prepare. It validates the request (job known, usability, setup type, argument check), creates the job directory <job-root>/<uuid> (and any --name alias), records the jobs row in the target lifecycle state with an empty native-cmd, emits the setup "used_by" provenance edge, resolves the scheduler options and backend, generates the batch script .job.sh, and freezes the resolved backend and options into a .submit metadata file so a later release (_knit_submit_dispatch) is deterministic and needs no re-resolution.

The four output arguments receive the job UUID, its directory, the --name alias symlink path (empty when no --name), and the job name. Callers MUST pass output variables whose names do not clash with this function's internal locals (uuid/jobdir/alias_link/job_name) — see the nameref-shadow-collision rule.

Output namerefs come first, then the target state, then the submit CLI args.

Parameters:

• __knit_ret1 -- [out] Out: the job UUID (the jobs row id).

• __knit_ret2 -- [out] Out: the job directory <job-root>/<uuid>.

• __knit_ret3 -- [out] Out: the --name alias symlink path (empty when none).

• __knit_ret4 -- [out] Out: the job name (the token after --).

• target_state -- [in] Lifecycle state to record ("submitted" or "prepared").

• ... -- [in] The submit CLI arguments (everything, including -- job args).

int _knit_submit()

Entry point for the submit CLI command. A submission is built and then dispatched: _knit_prepare_build validates the request, creates the job directory, records the jobs row, generates the batch script, and freezes the resolved backend and options into a .submit metadata file; _knit_submit_dispatch then issues the scheduler command. submit runs both back to back, so its behaviour is a build immediately followed by a release. prepare runs only the build (see prepare.sh).

Usage:

./exp.sh submit [--setup <setup-name>] [--name <alias>] [sched-args...] \
    -- job-name [args...]

int _knit_submit_cleanup_rejected()

Undo the eager bookkeeping of a submission the scheduler rejected, so a job that never ran leaves no trace. knit submit records the jobs row (and, for a job with a setup, a "used_by" provenance edge) and creates the job directory before dispatching, because a blocking --wait job needs the row to exist so the compute side can transition it. When the submission command itself fails, none of that should survive: this removes the jobs row, any provenance edge pointing at it, the --name alias symlink (if any), and the job directory. Each step is best-effort (the provenance table may not exist for a setup-less job).

Parameters:

• uuid -- [in] The submission's job UUID (its row id and edge target_id).

• jobdir -- [in] The job directory to remove.

• alias_link -- [in] Path to the --name alias symlink, or empty when none.

int _knit_submit_dispatch()

Release a built submission to the scheduler: the second phase of submit, and the whole of submit prepared / submit next. It reads the resolved backend and options from the job directory's .submit metadata (frozen by _knit_prepare_build), builds the scheduler submission command, records it as the row's native-cmd, advances the row to "submitted", issues the command, and handles a scheduler rejection (removing the never-run job, exactly as a direct submit does). On success it records the backend job id in .job.id.

Parameters:

• uuid -- [in] The job UUID (the jobs row id).

• jobdir -- [in] The job directory holding .submit and .job.sh.

• job_name -- [in] The job name (for log and error messages).

• alias_link -- [in] The --name alias symlink path, or empty when none (removed on rejection).

• wait_override -- [in] Optional: "true"/"false" to override the frozen "wait" option at release time (empty keeps the value frozen at build time). submit freezes --wait into .submit, so it passes nothing; submit prepared/submit next accept --wait at release and pass it here (prepared jobs freeze "wait" as false, since prepare has no --wait).

int _knit_submit_meta_read()

Restore the scheduler backend and resolved options a submission was built with from its .submit metadata file (see _knit_submit_meta_write). The backend name is written to the first named variable; each recorded option is written into the named associative array, which the caller declares.

Parameters:

• file -- [in] Path of the .submit file to read.

• out_backend -- [out] Name of the variable to receive the backend name.

• arr_name -- [out] Name of the associative array to populate with the options.

int _knit_submit_meta_write()

Freeze the resolved scheduler backend and options of a built submission into its .submit metadata file, so a later release reconstructs them without re-resolving. The format is one "key=value" per line: a single "backend=<name>" line, then one "opt:<key>=<value>" line per resolved option. A value is the rest of its line, so it may contain "=" or spaces but not a newline (no scheduler option does).

Parameters:

• file -- [in] Path of the .submit file to write.

• backend -- [in] Scheduler backend name.

• arr_name -- [in] Name of the resolved-options associative array to persist.

Variables

AssociativeArray _KNIT_JOBS

Associative array mapping registered job names to 1. Used to validate that a job names passed to knit submit is known.

String _KNIT_JOBS_TABLE

Name of the table recording every job and its lifecycle state. The row id is the job UUID; the "state" column moves submitted -> running -> completed, or -> killed when the scheduler terminates a job before it finishes. A submission the scheduler rejects never becomes a job and leaves no row at all (see _knit_submit_cleanup_rejected).

launch.sh

Functions

int _knit_launch_backend()

Resolve which launcher backend to use, by precedence:

per-run --launcher -> concrete launcher metadata -> KNIT_PROVIDED_LAUNCHER (a setup's frozen contract) -> none

Every launcher value is frozen ahead of the run — no run-time detection. The launcher metadata is the launcher integrated into the machine, resolved once at bootstrap from the profile or bootstrap-time detection; it is preferred over a setup's contract because a site's launcher cooperates with the resource manager as intended (§3 of the design). A "<unknown>" launcher (detection found no MPI at bootstrap, no profile) or an explicit "none" (the machine was told it offers no integrated launcher, via bootstrap --launcher none or a profile launcher.type of "none") is skipped so a providing setup's frozen KNIT_PROVIDED_LAUNCHER — set by knit_provides_launcher when the setup built or module-loaded its own MPI — is used instead. When nothing is configured, the "none" backend runs the app as a single rank-0 process. This mirrors how _knit_sched_backend degrades an undetected scheduler to the local background-process backend. Note that a run-time --launcher of "none" (the override argument, tier 1) is the opposite: an explicit, terminal choice of the "none" backend that does NOT fall through to a setup contract. Returns one of "none", "openmpi", "mpich", "pals", "slurm", "pbs", "flux".

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved backend name.

• override -- [in] Optional explicit launcher name (a per-run --launcher value); empty to fall back to the metadata then the setup contract.

int _knit_launch_bind_value()

The launcher layer is the app-side mirror of the scheduler layer (src/sched.sh). Where the scheduler places a job on an allocation, the launcher places the ranks of an app across the nodes of that allocation (MPI-style). A launcher is a separate axis from the scheduler: a Slurm job may run its ranks under OpenMPI's mpirun, under srun, etc. Each launcher backend lives in its own src/launch_<name>.sh with the same function set, dispatched here by name.

Backends:

• none — run the worker directly, no launcher: a single rank-0 process.

• openmpi — mpirun (Open MPI)

• mpich — mpiexec/Hydra (MPICH)

• slurm — srun (scheduler-integrated)

• pbs — the PBS mpiexec wrapper

• pals — mpiexec (HPE Cray PALS)

• flux — flux run (scheduler-integrated)

Translate knit's normalized --bind vocabulary into the spelling a given backend expects. knit defines a small set of portable binding units — none, core, socket, numa, thread — and each backend family renders them differently: Slurm's --cpu-bind uses the plural/locality-domain spellings (cores, sockets, ldoms, threads); PALS's --cpu-bind uses the singular spellings (core, socket, numa, thread); the OpenMPI/Hydra family (openmpi, mpich, pbs) matches PALS except that a hardware thread is spelled hwthread. Flux's -o cpu-affinity is a distribution policy (off | per-task | map:LIST), not a granularity, so all four levels map to per-task and none maps to off. An unrecognized value is not an error — it is passed through verbatim (with a warning) so a user can still reach a launcher-specific binding this vocabulary does not cover. Returns the translated value.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the translated value.

• backend -- [in] Launcher backend name ("openmpi", "mpich", "slurm", "pals", "flux").

• value -- [in] The knit --bind value to translate.

int _knit_launch_cmdline()

Dispatch to the configured backend's command-line builder, which fills a caller-provided array (passed by name) with the launcher argument vector (the launcher executable and its placement flags) for the resolved placement options. The "none" backend has no launcher, so it leaves the array empty.

Parameters:

• backend -- [in] Launcher backend name ("none", "openmpi", "mpich", ...).

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

• argv_name -- [in] Name of the array to fill with the launcher argument vector.

int _knit_launch_exec()

Dispatch to the configured backend's exec function, which runs the launcher with the translated placement flags followed by the worker command, and returns the launcher's exit status. The worker command is everything after a literal "--". The "none" backend runs the worker command directly, with no launcher.

Parameters:

• backend -- [in] Launcher backend name ("none", "openmpi", "mpich", ...).

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

launch_flux.sh

Functions

int _knit_launch_flux_cmdline()

The "flux" launcher backend places an app's ranks with Flux's flux run. Like the slurm/pbs backends — and unlike the MPI-native openmpi/mpich/pals backends — it is scheduler-integrated: flux run places tasks in the surrounding Flux instance directly. It is auto-detectable when a Flux instance owns the shell (FLUX_URI is set). flux run is task-based, which matches knit's normalized placement options directly:

--procs N -> -n N (--ntasks) --procs-per-node M -> -N <nnodes> (nnodes = ceil(procs / M)) --cpus-per-proc N -> -c N (--cores-per-task, native) --gpus-per-proc N -> -g N (--gpus-per-task, native) --hostnames h0,h1 -> --requires=host:h0,h1 --bind V -> -o cpu-affinity=<v> (V normalized by _knit_launch_bind_value) --gpu-bind V -> -o gpu-affinity=<V> (value passed through verbatim) --launcher-args … -> appended verbatim after the placement flags

flux run splits its placement flags into two families that must not be mixed: per-task (-n/-c/-g) and per-resource (--tasks-per-node/--gpus-per-node). knit keeps the per-task family so cpus-per-proc and gpus-per-proc stay native, and expresses procs-per-node as a node count (-N) instead of --tasks-per-node: with -n tasks over -N nodes, flux run fills the nodes evenly, so the ranks per node equal procs-per-node. nnodes is ceil(procs / procs-per-node).

flux run is a strong launcher fit: cpus-per-proc and gpus-per-proc are both native, where the Hydra-family backends warn and skip them. Binding is coarser: Flux's -o cpu-affinity is a distribution policy (off | per-task | map:LIST), not a granularity, so knit's core/socket/numa/thread all map to per-task (see _knit_launch_bind_value); reach exact placement through --launcher-args -o cpu-affinity=map:LIST.

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector. flux run executes inside the surrounding job's Flux instance and forwards its environment to every task, so the ranks inherit the job's KNIT_* variables and the setup env.

Build the launcher argument vector for the flux backend (the flux run executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set; any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_flux_exec()

Run flux run with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_flux_cmdline.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

launch_mpich.sh

Functions

int _knit_launch_mpich_cmdline()

The "mpich" launcher backend places an app's ranks with MPICH's mpiexec (the Hydra process manager). It translates knit's minimal placement options into Hydra flags:

--procs N -> -n N --procs-per-node M -> -ppn M --hostnames h0,h1 -> -hosts h0,h1 --bind V -> -bind-to <V> (V normalized by _knit_launch_bind_value) --launcher-args … -> appended verbatim after the placement flags

Hydra has no native flag for CPUs-per-rank (--cpus-per-proc) or GPU placement (--gpus-per-proc / --gpu-bind), so those are warned about and skipped; reach them through --launcher-args.

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector. Hydra forwards the submitting environment to every rank by default, so the ranks inherit the surrounding job's environment (KNIT_* variables and the setup env).

Build the launcher argument vector for the mpich backend (the mpiexec executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set; any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_mpich_exec()

Run mpiexec with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_mpich_cmdline.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

launch_none.sh

Functions

int _knit_launch_none_cmdline()

Build the launcher argument vector for the none backend into a caller-provided array, passed by name. There is no launcher, so the array is left empty. The placement is validated first so an over-specified request is still rejected on this path.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array.

int _knit_launch_none_exec()

Run the worker command directly, with no launcher, yielding a single rank-0 / size-1 process. The worker command is everything after a literal "--". The placement is validated first (single local rank only). Returns the worker's exit status.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

int _knit_launch_none_validate()

The "none" launcher backend is the graceful-degradation case: no MPI launcher is detected or configured, so an app runs as a single rank-0 / size-1 process directly on the host, with no launcher in front of it. It is not a standalone path — a run is always inside a job's allocation — it just means "this

allocation has no launcher, run one rank here."

Because it can only ever run one rank on the local host, it rejects any placement that asks for more: the only accepted request is a single local rank (procs 1, procs-per-node 1, hostnames = this host), or the equivalent with the options left to their defaults. Anything else (a second rank, a remote host) is a configuration error the user should see immediately rather than have silently collapsed to one local rank.

Fail fast unless the resolved placement is a single rank on the local host. The none backend has no launcher and cannot spread ranks, so a request for more than one process, more than one process per node, or a host other than this machine is fatal. Empty (defaulted) options are accepted — they resolve to the single local rank the backend runs. --launcher-args is ignored (there is no launcher to pass it to).

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames).

launch_openmpi.sh

Functions

int _knit_launch_openmpi_cmdline()

Build the launcher argument vector for the openmpi backend (the mpirun executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set; the --host value is slot-annotated by _knit_launch_openmpi_host_slots, and any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_openmpi_env_forward()

Build the mpirun "-x NAME" flags that forward the current environment to every rank, into a caller-provided array passed by name. mpirun does not forward the environment to ranks it starts on other nodes over SSH (see the file header), so the surrounding job's environment (KNIT_* variables and the setup env) is forwarded explicitly here to match srun/Hydra behavior.

Every exported variable is forwarded except two classes that must not be copied from the launching shell to the ranks:

• the launcher's own per-rank and resource-manager variables (OMPI_*, PMI_*, PMIX_*, and the Slurm/PBS/PALS/Flux/Hydra/PRTE families), which each rank or backend sets for itself — copying the launching shell's values would corrupt a rank's identity or confuse the backend; and

• shell- and host-local values (HOSTNAME, PWD, TMPDIR, SSH_* and the like), which are meaningful only on the launching node. The per-rank KNIT_MPI_RANK/SIZE/LOCAL_RANK are likewise skipped; the worker derives them from the launcher-native variables on each rank.

Parameters:

• argv_name -- [out] Name of the array to fill with the "-x NAME" flags.

int _knit_launch_openmpi_exec()

Run mpirun with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_openmpi_cmdline; the "-x" environment-forwarding flags (built by _knit_launch_openmpi_env_forward) are spliced in right after the mpirun executable so remote ranks inherit the job environment. The forwarding flags are intentionally left out of the recorded native command (which stays the clean placement command).

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

int _knit_launch_openmpi_host_slots()

The "openmpi" launcher backend places an app's ranks with Open MPI's mpirun. It translates knit's minimal placement options into mpirun flags:

--procs N -> -n N --procs-per-node M -> --npernode M --hostnames h0,h1 -> --host h0:S,h1:S (S = per-host slot count) --cpus-per-proc N -> --map-by slot:PE=N --bind V -> --bind-to <V> (V normalized by _knit_launch_bind_value) --launcher-args … -> appended verbatim after the placement flags

GPU placement (--gpus-per-proc / --gpu-bind) has no portable mpirun flag, so it is warned about and skipped; reach GPU affinity through --launcher-args. Note that --map-by slot:PE=N may conflict with --npernode (--procs-per-node), which is itself a mapping directive; this translation is best-effort (there is no live CI for it) and --launcher-args is the escape hatch if a site needs a different mapping.

The slot suffix on each --host entry is essential: mpirun's bare "--host h0,h1" advertises only one slot per host (a documented Open MPI default), so "-n 4" over two such hosts fails with "not enough slots" even inside a matching Slurm/PBS allocation, because an explicit --host overrides the resource manager's slot counts. Annotating each host with S slots (S = --procs-per-node when set, else ceil(procs / nhosts)) states the intended per-host capacity so the requested rank count fits. This is also what a plain SSH/none-scheduler cluster needs, where there is no resource manager to supply slots at all.

Exception — Open MPI built with PBS TM integration (--with-tm): inside a PBS job Open MPI's tm RAS already knows the allocated nodes and their slot counts, and an explicit --host (with or without slot suffixes) conflicts with it — the mapper rejects the placement ("requested more processes than the ppr ... can

support", or "not enough slots"). So when $PBS_NODEFILE is set the --host flag is omitted entirely and the allocation is left to TM; -n / --npernode still control the rank count and per-node distribution. (Slurm's PLM, by contrast, tolerates the redundant --host and needs it to honour a strict host subset, so it is kept there.) A strict host

subset of a PBS allocation via --hostnames is therefore not expressible to this backend; use MPICH there.

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector.

Environment forwarding: unlike srun and MPICH's Hydra, which pass the launching environment to every rank, mpirun forwards nothing to ranks it starts on other nodes over SSH. The ranks on the launching node inherit the job's environment by fork, but the remote ranks would start with a bare login environment and lose the surrounding job's environment (KNIT_* variables and the setup env). So _knit_launch_openmpi_exec forwards the environment explicitly with mpirun's -x, skipping the variables that must stay per-rank or per-host (the launcher's own rank/PMI variables and shell-local values); see _knit_launch_openmpi_env_forward.

Build the value of mpirun's --host flag from a comma-separated host list, annotating each host with a per-host slot count so the requested rank count fits (see the file header for why bare --host is not enough). The slot count is --procs-per-node when set; otherwise it is derived as ceil(procs / nhosts) so procs ranks spread across the hosts. When neither procs-per-node nor procs is known, the hosts are emitted unannotated (nothing to size them by).

Parameters:

• hosts -- [in] Comma-separated host list (the resolved --hostnames value).

• ppn -- [in] Resolved --procs-per-node value (may be empty).

• procs -- [in] Resolved --procs value (may be empty).

launch_pals.sh

Functions

int _knit_launch_pals_cmdline()

The "pals" launcher backend places an app's ranks with the HPE Cray PALS (Parallel Application Launch Service) mpiexec, as found on ALCF Polaris and Aurora. Like openmpi and mpich — and unlike the scheduler-integrated slurm/pbs backends — it is MPI-native and auto-detectable (_knit_detect_launcher recognises PALS by its mpiexec --help banner). It translates knit's minimal placement options into PALS mpiexec flags:

--procs N -> -n N --procs-per-node M -> --ppn M --hostnames h0,h1 -> --hosts h0,h1 --cpus-per-proc N -> --depth N --bind V -> --cpu-bind <V> (V normalized by _knit_launch_bind_value) --launcher-args … -> appended verbatim after the placement flags

PALS uses the long spellings --ppn and --hosts (double dash), distinguishing it from the Hydra-based mpich/pbs backends that use -ppn/-hosts. To bind each rank to the cores reserved by --depth, pass --bind (e.g. --bind core) or --launcher-args "--cpu-bind depth"; knit does not auto-inject a --cpu-bind. GPU placement (--gpus-per-proc / --gpu-bind) has no PALS mpiexec flag — GPU affinity on PALS systems is set by a wrapper script — so it is warned about and skipped; reach it through --launcher-args.

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector. PALS mpiexec forwards the submitting environment to every rank by default, so the ranks inherit the surrounding job's environment (KNIT_* variables and the setup env).

Build the launcher argument vector for the pals backend (the mpiexec executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set; any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_pals_exec()

Run the PALS mpiexec with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_pals_cmdline.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

launch_pbs.sh

Functions

int _knit_launch_pbs_cmdline()

The "pbs" launcher backend places an app's ranks with the PBS mpiexec wrapper (Hydra-based, as shipped with PBS Pro / OpenPBS). Unlike the MPI-native backends (openmpi/mpich), it is scheduler-integrated: the PBS mpiexec wrapper reads $PBS_NODEFILE from the surrounding allocation. It is never auto-detected — it is selectable only via --launcher pbs, the launcher metadata, or a machine profile. It translates knit's minimal placement options into Hydra flags:

--procs N -> -n N --procs-per-node M -> -ppn M --hostnames h0,h1 -> -hosts h0,h1 --bind V -> -bind-to <V> (V normalized by _knit_launch_bind_value) --launcher-args … -> appended verbatim after the placement flags

The Hydra-based wrapper has no native flag for CPUs-per-rank (--cpus-per-proc) or GPU placement (--gpus-per-proc / --gpu-bind), so those are warned about and skipped; reach them through --launcher-args.

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector. The wrapper runs inside the surrounding job's allocation and forwards its environment to every rank, so the ranks inherit the job's KNIT_* variables and the setup env.

Build the launcher argument vector for the pbs backend (the mpiexec executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set; any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_pbs_exec()

Run the PBS mpiexec wrapper with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_pbs_cmdline.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

launch_slurm.sh

Functions

int _knit_launch_slurm_cmdline()

The "slurm" launcher backend places an app's ranks with Slurm's srun. Unlike the MPI-native backends (openmpi/mpich), it is scheduler-integrated: srun talks to the surrounding Slurm allocation directly. It is never auto-detected — it is selectable only via --launcher slurm, the launcher metadata, or a machine profile. It translates knit's minimal placement options into srun flags:

--procs N -> --ntasks N --procs-per-node M -> --ntasks-per-node M --hostnames h0,h1 -> --nodelist h0,h1 --nodes k (k = number of hosts) --cpus-per-proc N -> --cpus-per-task N --bind V -> --cpu-bind=<V> (V normalized by _knit_launch_bind_value) --gpus-per-proc N -> --gpus-per-task N --gpu-bind V -> --gpu-bind=V (value passed through verbatim) --launcher-args … -> appended verbatim after the placement flags

Placement is resolved and validated upstream by the run dispatcher; this backend only formats the resolved triple into an argument vector. srun runs inside the surrounding job's allocation and forwards its environment to every task, so the ranks inherit the job's KNIT_* variables and the setup env.

Build the launcher argument vector for the slurm backend (the srun executable followed by its placement flags) into a caller-provided array, passed by name. Each placement flag is added only when the corresponding option is set. When --hostnames is given, the node count for --nodes is derived from the number of comma-separated hosts. Any --launcher-args string is word-split and appended verbatim.

Parameters:

• argv_name -- [out] Name of the array to fill with the launcher argument vector.

• opts_name -- [in] Name of the resolved placement-options associative array (keys: procs, procs-per-node, hostnames, launcher-args).

int _knit_launch_slurm_exec()

Run srun with the translated placement flags followed by the worker command, and return its exit status. The worker command is everything after a literal "--". The launcher argv is built by _knit_launch_slurm_cmdline.

Parameters:

• arr_name -- [in] Name of the resolved placement-options associative array.

• -- -- [in] Literal separator.

• ... -- [in] The worker command and its arguments.

local.sh

Functions

int _knit_submit_local()

Submit a command as a background process, acting as a minimal local job scheduler for development and testing on machines without an HPC scheduler.

The process is launched with nohup so it survives terminal disconnects. nohup exec-replaces itself with the target command without forking, so the returned PID refers directly to the running command process.

Prints the PID of the background process on stdout and returns 0. Returns 1 on argument errors.

Everything after a literal -- is the command (and its arguments) to run and is not validated as a submission option. Options may be given either as "--name value" or "--name=value".

Parameters:

• ... -- [in] Submission options followed by -- and the command to run: --stdout <file> Redirect command stdout to <file> (default: /dev/null) --stderr <file> Redirect command stderr to <file> (default: /dev/null) --stdin <file> Redirect command stdin from <file> (default: /dev/null) --walltime HH:MM:SS Kill the command after this wall-clock time.

int _knit_wait_local()

Wait until a locally submitted background process has finished.

Uses kill -0 polling because wait(1) only works for child processes of the current shell, and _knit_submit_local detaches the process via nohup.

Parameters:

• pid -- [in] PID returned by _knit_submit_local.

int _knit_walltime_to_seconds()

Convert a wall-clock time string in HH:MM:SS format to an integer number of seconds.

Parameters:

• walltime -- [in] Wall-clock time in HH:MM:SS format.

log.sh

Functions

int _knit_ensure_trace_file()

Create the trace file on first use and cache its path in _KNIT_TRACE_FILE. Must be called in the current shell (not a subshell) before reading _KNIT_TRACE_FILE.

int _knit_log()

This function acts like printf but takes a log level first, and adds [knit:<level>] in front and

after the text. It outputs to stderr.

Example:

_knit_log info "Hello, Matthieu"

Parameters:

• level -- [in] Logging level.

• ... -- [in] Arguments for printf.

int _knit_log_level_to_int()

Convert a log level string to its integer value, storing it in the caller-named variable. trace=0, debug=1, info=2, warning=3, error=4, critical=5. Yields 2 (info) for unrecognized values.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the integer value.

• level -- [in] Log level string.

Variables

String _KNIT_TRACE_FILE

The trace file is the file used to redirect the output of programs. It is created lazily by _knit_ensure_trace_file() on first use rather than eagerly here: creating it at source time ran an mktemp on every "source knit.sh" and leaked a temporary file each time (the test suite sources knit.sh hundreds of times). Empty until first needed.

main.sh

Variables

ExportedString _KNIT_JUMP_TO_DIR

Opt-in, internal environment variable used by the re-entry paths (submit, run) to relocate the shell after the framework has been sourced. Those paths arrange for the experiment to be sourced from the directory holding knit.sh (so a bare source knit.sh resolves) and set this variable to the directory the body should actually run in. When it is unset (the normal top-level invocation) nothing happens: the feature is fully inert for ordinary runs.

prepare.sh

Functions

int _knit_prepare()

Entry point for the prepare CLI command. Builds a submission and records it with state "prepared" without contacting the scheduler (see _knit_prepare_build): the batch script and the frozen .submit metadata are written, but no scheduler command is issued and no .job.id is recorded. The job stays queued until released by submit prepared or submit next. Prints the job UUID (the canonical, scheduler-independent identifier).

Usage:

./exp.sh prepare [--setup <setup-name>] [--name <alias>] [--group <name>] \
    [sched-args...] -- job-name [args...]

int _knit_prepare_claim_id()

Atomically claim one prepared job by its UUID, moving it from state "prepared" to the transient "submitting". Like _knit_prepare_claim_next but keyed on an explicit id: a single conditional UPDATE under the write lock whose guard ("AND state='prepared'") both prevents a double claim and yields nothing when the row is no longer prepared. Prints the claimed UUID, or nothing when the row is absent or not in state "prepared".

Parameters:

• uuid -- [in] The job UUID to claim.

int _knit_prepare_claim_next()

Atomically claim the oldest prepared job matching optional filters, moving it from state "prepared" to the transient "submitting" so no concurrent releaser can grab the same row. The whole pick-and-mark is one conditional UPDATE under the write lock (see _knit_sqlite3_write): the inner SELECT picks the lowest id in state "prepared" (job UUIDs are time-ordered uuidv7, so lowest id is oldest prepared), and the outer guard re-checks the state so a row another releaser claimed first is not taken twice. Job UUIDs order by prepare time, so this releases in prepare order. Prints the claimed UUID, or nothing when no prepared job matches (queue drained, or the race was lost).

Parameters:

• job -- [in] Optional job-name filter (the "job" column); empty means any.

• group -- [in] Optional group filter (the "group" column); empty means any.

int _knit_prepare_release()

Release a claimed prepared job (already moved to state "submitting" by one of the claim helpers) to the scheduler. The submission spec was frozen at prepare time, so this reconstructs the job name and any --name alias from the jobs row, then hands off to _knit_submit_dispatch, which builds the submit command, advances the row "submitting" -> "submitted", and issues it (cleaning up on a scheduler rejection exactly as a direct submit does). Prints the released job's UUID.

Parameters:

• uuid -- [in] The claimed job UUID.

• wait_flag -- [in] "true"/"false": block until the job completes (see _knit_submit_dispatch's wait override).

int _knit_prepare_remove()

Remove a prepared job that was never dispatched: delete its jobs row, its job directory, and any --name alias. A prepared job has contacted no scheduler, so there is nothing to cancel; the teardown is exactly the rejection cleanup a failed submit performs (_knit_submit_cleanup_rejected), reached here by the "prepared" branch of job cancel. The --name alias is reconstructed from the row's "name" column, as _knit_prepare_release reconstructs it on dispatch.

Parameters:

• uuid -- [in] The prepared job UUID.

profile.sh

Functions

int _knit_load_profile()

Extract all portable fields from a profile's JSON and store them in global variables. Called by bootstrap after jq is available, with the resolved JSON content (not a name).

Sets (empty string when a field is absent): _KNIT_PROFILE_SCHEDULER_TYPE _KNIT_PROFILE_SCHEDULER_COMMAND _KNIT_PROFILE_SCHEDULER_DEFAULT_QUEUE _KNIT_PROFILE_SCHEDULER_DEFAULT_ARGS (space-joined from JSON array) _KNIT_PROFILE_LAUNCHER_TYPE _KNIT_PROFILE_LAUNCHER_COMMAND _KNIT_PROFILE_LAUNCHER_DEFAULT_ARGS (space-joined from JSON array) _KNIT_PROFILE_CORES_PER_NODE _KNIT_PROFILE_GPUS_PER_NODE

Parameters:

• json -- [in] The resolved profile JSON content.

int _knit_profile_admin_entries()

List the admin-provided profiles under _KNIT_PROFILE_ADMIN_DIR as "<name><TAB><description><TAB><hidden>" lines, one per profile, where <name> is the path relative to the admin directory minus the .json suffix and <hidden> is "true" when the profile sets "_hide": true (mirroring the GitHub index's hidden field). Empty when the directory is absent.

Every profile is listed, including hidden ones; knit_list_profiles does the filtering. Both the hidden check and the description read are jq-free (a grep/sed for the marker) so listing keeps working before bootstrap.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the newline-separated entries.

int _knit_profile_file_description()

Extract a profile file's description field jq-free, so knit profile list can show it before bootstrap (jq may be absent). Uses a sed for the first "description": "..." occurrence; relies on the value containing no literal '"'. The result is empty when the field is absent.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the description.

• file -- [in] Path to the profile JSON file.

int _knit_profile_github_url()

Build the raw.githubusercontent.com URL for a profile path at a ref. Uses the raw host (not github.com/.../blob/...) so the response is the JSON itself, not an HTML page.

Parameters:

• path -- [in] Profile path under src/profiles, e.g. "anl/improv".

• ref -- [in] Git ref (tag, branch, or SHA).

int _knit_profile_http_get()

Fetch a URL and, on HTTP 200, store the body in the named variable. Records the HTTP status in _KNIT_PROFILE_LAST_HTTP. Returns 0 only on HTTP 200.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the response body.

• url -- [in] URL to fetch.

int _knit_profile_is_hidden()

Return success when the profile file names its _hide field as true. Uses a jq-free grep so it works before bootstrap (jq may be absent), tolerating the usual JSON spacing around the colon.

Parameters:

• file -- [in] Path to the profile JSON file.

int _knit_profile_latest_ref()

Resolve the latest knit release tag via the GitHub API (mirrors _knit_spack_latest_release). Prints the tag; fatal if none can be resolved.

int _knit_profile_list()

int _knit_profile_parse_index()

Parse a profile index into one "<name><TAB><description><TAB><hidden>" line per entry. The index is a JSON array of one-line objects, e.g. { "name": "anl/aurora", "description": "...", "hidden": true } (see gen-profile-index.sh). Extraction is jq-free (a single sed) so it works before bootstrap, when jq is not yet available; it relies on the description containing no literal '"', which the generator guarantees for shipped descriptions.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the newline-separated list.

• body -- [in] The index.json content.

int _knit_profile_show()

Implementation of 'knit profile show'. For a bootstrapped experiment, prints the profile frozen at bootstrap; otherwise resolves the given spec (URL / local file / /etc/knit / shorthand) and prints it.

int _knit_render_platform_files()

Materialize the profile's platform artifacts under _KNIT_PREFIX: platform.sh (modules + environment) and spack-config.json (the profile's spack object). Either file is left absent when the profile omits the corresponding fields. Called by bootstrap after the profile is resolved.

Parameters:

• json -- [in] The resolved profile JSON content.

int _knit_render_platform_sh()

Render the platform shell fragment (§5.3) to a file: an optional module-init source line, an optional module purge, a single module load of the profile's modules, and one export KEY=VALUE per environment entry. The file is left absent (not created) when the profile has neither modules nor environment. Fatal when modules is present but no module init resolves.

Parameters:

• json -- [in] The resolved profile JSON content.

• outfile -- [in] Path of the platform.sh file to write.

int _knit_render_spack_config()

Render the profile's spack object as a single Spack environment-config fragment. The keys under spack are Spack config section names (packages, mirrors, concretizer, config, compilers, ...) and each value is that section's content exactly as it appears under the corresponding top-level key in a Spack config file. Knit does not interpret the contents: it writes the whole object back wrapped under a top-level spack key, i.e. { "spack": <value> }, which is the Spack environment-manifest form. Because JSON is a subset of YAML, Spack ingests the file directly: it is spack config added into the environment (always -e <env>) at env-install time, where the spack wrapper lets a single file carry every section at once. Nothing is written when the profile declares no spack object (or an empty one).

Parameters:

• json -- [in] The resolved profile JSON content.

• outfile -- [in] Path of the spack-config file to write.

int _knit_resolve_module_init()

Resolve the environment-module init script to source when materializing the platform (§5.2), trying in order: an explicit profile "module_init" override; the MODULESHOME-derived path and the standard candidate list (_KNIT_MODULE_INIT_CANDIDATES); finally, if module is already a function or command in this environment, no init line is needed (empty result). Returns non-zero when none applies, so the caller can fatal.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the resolved init path (empty when module is already available and no script is needed).

• json -- [in] The resolved profile JSON content.

int _knit_resolve_profile()

Resolve a --profile <spec> to its JSON content and a canonical label, trying each source in order (first match wins):

1. URL (http:// or https://) fetched verbatim

2. local file <spec> or <spec>.json on disk (the offline story)

3. admin profile /etc/knit/profiles/<spec>.json

4. GitHub shorthand <namespace>/<machine>[/<variant>...][<ref>]; the path may be two or more segments (e.g. nersc/perlmutter/cpu). A bare ref defaults to the default branch (most up-to-date profiles); @latest resolves via the releases API

On success sets the JSON content and the resolved label (a URL, path, or "<path>@<ref>"). If nothing resolves, fatal with a message enumerating every source that was tried.

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the resolved JSON content.

• __knit_ret2 -- [out] Name of the variable to hold the resolved label.

• spec -- [in] The --profile argument.

Variables

Array _KNIT_MODULE_INIT_CANDIDATES

Ordered list of environment-module init scripts searched (after an optional profile "module_init" override and the MODULESHOME-derived path) when materializing .knit/platform.sh. The first that exists is sourced to make the module shell function available. Overridable for testing.

String _KNIT_PROFILE_ADMIN_DIR

Directory of admin-provided (site) profiles, tried before the GitHub store so a machine's own copy wins and resolves offline. Overridable for testing.

String _KNIT_PROFILE_DEFAULT_REF

Git ref the GitHub shorthand (and the profile index) resolve against when no explicit is given. This is the default branch, not the running knit version tag, because the profile store on the default branch carries the most up-to-date machine profiles.

String _KNIT_PROFILE_LAST_HTTP

HTTP status of the most recent _knit_profile_http_get call ("200" on success, the returned status otherwise, empty when curl itself failed). Used to build the enumerating "profile not found" error.

String _KNIT_PROFILE_REPO

GitHub repository serving the in-repo profile store, and the org/repo the shorthand form resolves against.

prov.sh

Functions

int _knit_produced_edge_sql()

Build (print, without executing) the INSERT for a "produced" provenance edge: a producing invocation (the source) produced an artifact (the target, a row of the artifacts table). It is a thin wrapper over _knit_prov_edge_sql that bakes in the produced-edge shape: the target_name is the artifacts node label, the edge_type is "produced", and — a produced edge has no duration and no call site — the two timestamps and the alias are NULL. Meant to be composed into the producing invocation's record-time transaction (see the artifacts write path).

Parameters:

• source_id -- [in] UUID of the producing invocation (empty for a root).

• source_name -- [in] Demangled command name of the producer.

• artifact_id -- [in] UUID of the produced artifact (the artifacts row id).

int _knit_prov_create_table()

Create the provenance edge table if it does not already exist. Each row is one directed relationship, "source --edge_type--> target", of one of three kinds: a "call" edge (source invoked target), a "used_by" edge (target references a setup or resource, which is the source, built by an earlier invocation), or a "produced" edge (source invocation produced target, an artifacts-table row). The source is always the antecedent (the caller, the setup, the producer) and the target the dependent (the callee, the consumer, the artifact). Node identity is the pair (id, name); the timestamps are REAL epoch seconds and are NULL for "used_by" and "produced" edges. The nullable "alias" column holds the call-site name recorded by knit_as (NULL for a plain edge). Called at bootstrap alongside the metadata table.

int _knit_prov_edge_sql()

Build (print, without executing) the INSERT statement for one provenance edge. The statement is meant to be run through _knit_sqlite3_write, either on its own (see _knit_prov_record_edge) or inside a transaction next to a data-row insert (see _knit_db_record_invocation). Timestamps are rendered as NULL when empty.

Parameters:

• source_id -- [in] UUID of the source (caller for "call"; setup for "used_by"); empty for a root invocation.

• source_name -- [in] Demangled command name of the source (empty for a root).

• target_id -- [in] UUID of the target (callee for "call"; consumer for "used_by").

• target_name -- [in] Demangled command name of the target.

• edge_type -- [in] "call" (source invoked target), "used_by" (target references a setup/resource, which is the source), or "produced" (source produced target, an artifact).

• start_time -- [in] Epoch seconds when the call started (empty -> NULL).

• end_time -- [in] Epoch seconds when the call returned (empty -> NULL).

• alias -- [in] Call-site name from knit_as (empty -> NULL).

int _knit_prov_ensure_table()

Ensure the provenance edge table exists before an edge is written, creating it lazily on first use. A freshly bootstrapped experiment already has the table (created at bootstrap), but a database bootstrapped before this feature shipped does not; ensuring it here lets a new invocation record edges (and keeps its data-row-plus-edge transaction from rolling back) rather than failing. The create is idempotent and runs at most once per process, guarded by _KNIT_PROV_TABLE_ENSURED.

int _knit_prov_now()

Print the current time as a REAL number of seconds since the Unix epoch, at the best precision available. Used for a call edge's start_time (captured when the frame is pushed) and end_time (captured at record time), so a duration is a plain subtraction.

int _knit_prov_nullable_literal()

Render an argument as a SQL literal for a nullable column: an empty argument becomes a bare NULL; a non-empty argument becomes a single-quoted, escaped literal. Used for the two REAL timestamp columns (empty for "used_by" edges, which have no duration; SQLite's type affinity coerces the quoted number to a REAL) and for the TEXT "alias" column (empty for a plain, unaliased edge).

Parameters:

• value -- [in] Column value, or empty for NULL.

int _knit_prov_record_edge()

Insert a single provenance edge into the edge table, serialized through the advisory-locked writer. Used on its own for a target that records no data row (a table-less command) and for "used_by" edges; a target that also records a data row writes both in one transaction via _knit_db_record_invocation instead.

Parameters:

• source_id -- [in] See _knit_prov_edge_sql.

• source_name -- [in] See _knit_prov_edge_sql.

• target_id -- [in] See _knit_prov_edge_sql.

• target_name -- [in] See _knit_prov_edge_sql.

• edge_type -- [in] See _knit_prov_edge_sql.

• start_time -- [in] See _knit_prov_edge_sql.

• end_time -- [in] See _knit_prov_edge_sql.

int _knit_record_used_by_edge()

Record a "used_by" provenance edge from an already-resolved source node (a setup or a resource instance) to a consuming invocation (the target). A "used_by" edge has no duration, so both timestamps are NULL. Shared by the setup-dependency after-callback (setup:<type> source) and the resource- dependency after-callback (resource:<type> source); each caller reads its own on-disk id/type markers and passes the resolved node identity here.

Best-effort and gated with the other provenance writes: it records nothing when recording is disabled, on a suppressed rank, before bootstrap, when the target does not participate in the graph, or when the source id is empty (e.g. a setup or resource materialized before provenance shipped).

Parameters:

• source_id -- [in] Resolved row id of the source node (empty -> no edge).

• source_name -- [in] Node name of the source ("setup:<type>" / "resource:<type>").

• target_cmd -- [in] Mangled command name of the consumer (the edge target).

• target_id -- [in] Resolved row id of the consumer (the edge target).

Variables

String _KNIT_PROV_TABLE

Name of the framework-owned edge table that records the relationships between invocations (the provenance graph). The "__" prefix/suffix marks it as reserved (not a user-provided command table), like other reserved names.

String _KNIT_PROV_TABLE_ENSURED

Set to "1" once the provenance table has been ensured in this process (see _knit_prov_ensure_table), so the idempotent CREATE runs at most once per run.

query.sh

Functions

int _knit_query_annotate_catalog()

Filter the raw query catalog listing read from standard input, annotating it with two things read from the live schema: each table <name> line whose table has a distinct command-name alias gains " (command: <name>)" so users discover both spellings of a label, and each column <name> line gains " (<TYPE>)" with the column's SQL storage type. Column types are looked up once per table (through the stubbable _knit_query_column_types helper) as the table line is seen. A TABLE.COLUMN validation line (a single line with no table/column prefix) is passed through unchanged.

int _knit_query_catalog_columns()

int _knit_query_catalog_graph_tables()

Print the names of every graph table in the database, sorted by name. A table is enumerated from sqlite_master (user tables only) and kept when _knit_query_catalog_is_graph_table accepts it.

int _knit_query_catalog_has_column()

Return success when the table has a column of the given name.

Parameters:

• table -- [in] The table name.

• want -- [in] The column name to look for.

int _knit_query_catalog_is_graph_table()

Return success when the named table exists and participates in the graph: it is the edge table __provenance__, or it has an id column (the uuid7 key that ties node rows to edges). This mirrors the node/edge classification the transpiler applies, so the catalog lists exactly the queryable entities.

Parameters:

• table -- [in] The table name to test.

int _knit_query_catalog_print_table()

Print one table's listing in the raw catalog format the annotator consumes: a table <name> line followed by one column <name> line per column.

Parameters:

• table -- [in] The table name to print.

int _knit_query_catalog_produce()

Produce the raw (un-annotated) catalog listing on standard output. With no reference, list every graph table and its columns, sorted by name. With a TABLE reference, list that one table. With a TABLE.COLUMN reference (split at the last dot), print the reference when the column exists. An unknown table or column is reported to stderr and returns non-zero, so the command exits with a failure status just as the engine's --catalog mode did.

Parameters:

• ref -- [in] Empty to list all; else TABLE or TABLE.COLUMN.

Returns:

Non-zero on an unknown table/column reference.

int _knit_query_read_output_opts()

Read the OUTPUT-OPTS shared by knit query graph and knit query sql out of a command invocation into three caller-named variables. Factored here so both query commands parse --format/--header/--separator identically; both then translate the values into sqlite3 dot-commands (the lens runs every query through sqlite3). The format defaults to list and the header defaults OFF (query output is most often piped elsewhere, where a header is noise); the separator defaults to empty (the output mode's own default).

Parameters:

• __knit_ret1 -- [out] Name of the variable to hold the format value.

• __knit_ret2 -- [out] Name of the variable to hold the header flag ("true"/"false").

• __knit_ret3 -- [out] Name of the variable to hold the separator value.

• ... -- [in] The command invocation arguments to read the options from.

int _knit_query_table_alias()

Return, through a caller-named variable, the command name a table is registered under when it differs from the table name, or the empty string otherwise. A command that overrides its table with knit_with_table (e.g. submit -> jobs, submit:montecarlo -> montecarlo) has a distinct command-name alias; a command whose table is its own name (setups, downloads, plain commands) has none. Read from the live registration state (_KNIT_DB_REGISTERED_TABLES), so it can never go stale.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the alias (empty if none).

• table -- [in] The table name to look up.

remove.sh

Functions

int _knit_remove_append_ids()

Run a read-only SELECT and append each non-empty result line to a caller-named array. Errors (e.g. a table that was registered but never created) are silenced and simply yield no rows, so a probe against a not-yet-materialized table is a no-op rather than a failure. The array is appended to, never reset, so several calls can accumulate matches (used when a selector spans more than one table).

Parameters:

• __knit_ret -- [out] Name of the array to append the selected ids to.

• sql -- [in] The SELECT statement to run.

int _knit_remove_build_report()

Assemble the itemized removal report from the erase set and its maps, filling four caller-named arrays that _knit_remove_print_report lays out. All database reads and formatting happen here, so the printer is pure layout:

• the data-row lines, one per erase-set id in order, each "<kind> <label>

  <id>" with a short annotation. A setup or resource shows "<type> (<name>)"; a job submission shows its state; a job body row is marked as such; a run shows its launched app; an app or plain command shows its command name; an artifact shows its path.

• the provenance-edge lines: every provenance edge with an endpoint in the set (exactly the edges the deletion removes, including a kept provider's used_by edge into an erased consumer).

• the on-disk directories and artifact entries the removal deletes: when keep_mode is not "files", each job directory and each setup/resource instance directory, and -- when keep_mode is "none" -- each artifact entry under the artifact root. Under "files" this list is empty: the removal deletes nothing on disk.

• the "left on disk" lines: the plain non-artifact outputs (always, attributed to the owning command); when keep_mode is "artifacts", the kept artifact entries; and when keep_mode is "files", every job/setup/resource directory and artifact entry too, each tagged by kind (nothing is removed).

Parameters:

• __knit_ret1 -- [out] Name of the array to fill with data-row lines.

• __knit_ret2 -- [out] Name of the array to fill with provenance-edge lines.

• __knit_ret3 -- [out] Name of the array to fill with removed dir/entry paths.

• __knit_ret4 -- [out] Name of the array to fill with "left on disk" lines.

• __knit_tables -- [in] Name of the assoc array mapping id -> table.

• __knit_kinds -- [in] Name of the assoc array mapping id -> kind.

• __knit_apaths -- [in] Name of the assoc array mapping artifact id -> path.

• __knit_plain -- [in] Name of the assoc array mapping plain output path -> command.

• keep_mode -- [in] Removal policy: "none" removes everything, "artifacts" keeps artifact entries, "files" keeps the whole tree.

• ... -- [in] The erase-set ids, in order.

int _knit_remove_check_refusal()

Enforce the callee/artifact refusal for the default (downward) closure. For each originally selected id, walk the incoming "call"/"produced" edges BACKWARD from it and refuse the whole operation on reaching a caller or producer that is kept (not in the erase set) AND owns a data row -- a persistent entity whose recorded provenance would otherwise be left referring to a row that is gone. Erasing a callee while such a caller stays (or an artifact while its producer stays) is rejected.

The walk passes THROUGH table-less dispatcher frames: the "setup", "fetch", and "submit" dispatchers each record a "call" edge into the body row they create but own no table of their own, so such a frame is not itself a kept caller and the walk continues up from it. A branch that reaches the root (an empty source) or an ancestor already in the erase set first is not a refusal. This is what lets "remove setup"/"remove resource" work by default: the only "call" edge into a freshly built setup or resource body comes from its dispatcher, above which sits the root. It still refuses when a user's own table-backed command (e.g. one that itself calls "knit setup" to build several setups) is the kept caller, because that command opted into provenance by declaring a table -- the walk climbs past the dispatcher to it and points the user at removing THAT command or passing --from-root. A table-less helper command, by contrast, is walked through like a dispatcher, so setups it created can be removed independently.

A "call" refusal points the user at removing the caller (with the caller's own remove subcommand) or passing --from-root; a "produced" refusal (a bare remove:artifact) points at --from-root, which pulls the producer in and erases the whole lineage.

Parameters:

• __knit_selected -- [in] Name of the array of originally selected ids.

• __knit_erase -- [in] Name of the array holding the full erase set.

Returns:

Fatal on the first refusal; otherwise 0.

int _knit_remove_check_terminal_jobs()

Enforce that every job in the erase set has finished before anything is deleted. For each erase-set id whose table is the jobs table (read from the id -> table map _knit_remove_map_ids produced), the job's "state" column is read; a state other than "completed" or "killed" (that is, "submitted", "running", or "prepared") refuses the whole operation and points the user at "job cancel <id>" to stop or tear down the job first. There is no --force override: a live job must be cancelled before it can be erased. The refusal fires whether the job was named directly or pulled into the set by a cascade (a setup or resource it used being erased), because both reach it through the same erase-set membership.

Parameters:

• __knit_tables -- [in] Name of the assoc array mapping id -> table.

• ... -- [in] The erase-set ids.

Returns:

Fatal on the first non-terminal job; otherwise 0.

int _knit_remove_closure_downward()

Compute the default downward erase set from a set of starting ids, returned through a caller-named array. Starting from each id, the closure follows every outgoing provenance edge (source -> target) of the three edge types to a fixed point: a "call" edge (a caller owns its callees), a "produced" edge (a producer owns its artifacts), and a "used_by" edge from a provider (a setup or resource owns its consumers). Because the walk only follows edges where the current id is the SOURCE, a "used_by" edge is never followed backward into a provider (which would be the target), so erasing a consumer leaves its setup/resource intact. An associative visited set guards against revisiting an id (and against any accidental cycle from bad data), so each reachable id appears once.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the erase-set ids.

• ... -- [in] The starting ids.

int _knit_remove_closure_from_root()

Compute the whole-lineage erase set that --from-root selects, returned through a caller-named array: the connected component of the starting ids in the subgraph made of "call" and "produced" edges only, traversed in BOTH directions. From each id the walk adds every neighbour across a call/produced edge, backward (target -> source: the caller that invoked it, or the producer that made it) and forward (source -> target: everything it called or produced). Iterated to a fixed point, the two directions climb to the root of the tree and back down over the whole tree, so pointing at any row (a callee, a run, an app, or an artifact) names the same lineage. "used_by" edges are NEVER queried, in either direction, so a setup or resource used by the tree is left intact (it belongs to its own tree). An associative visited set guards against revisiting an id, so each id appears once. There is no refusal check in this mode: pulling in the kept caller/producer is exactly what --from-root is for.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the erase-set ids.

• ... -- [in] The starting ids.

int _knit_remove_confirm()

Print the removal report and decide whether the erase may proceed. The report is always printed, so a run always leaves a record of what it did (or would do); the header tense matches the mode. With yes == "true" the past-tense header "Erased:" is printed and the function returns success without prompting. Otherwise the future-tense header "The following will be permanently erased:" is printed and the user is prompted: one line is read (read -r), and only "y", "Y", or "yes" proceeds; anything else (a bare Enter is the default No) prints "Aborted." and returns failure, which the caller treats as a clean decline (exit 0), not an error. When stdin is NOT an interactive terminal and yes was not given, the prompt cannot be answered, so rather than block on a read that never returns the function is a fatal refusal that tells the user to pass --yes for non-interactive use. The four report arrays are passed BY NAME and forwarded verbatim to _knit_remove_print_report.

Parameters:

• yes -- [in] "true" to skip the prompt (the report is still printed).

• rows_name -- [in] Name of the array of data-row lines.

• edges_name -- [in] Name of the array of provenance-edge lines.

• removed_name -- [in] Name of the array of removed dir/entry paths.

• left_name -- [in] Name of the array of "left on disk" lines.

Returns:

0 to proceed; 1 on an interactive decline. Fatal on a non-terminal stdin without --yes.

int _knit_remove_declare_flags()

Declare the flags shared by every remove subcommand. Call it between knit_register and knit_done.

int _knit_remove_declare_selectors()

The remove command group erases recorded entities (a setup, a resource, a job, a run, an app invocation, a plain command invocation, or an artifact) together with their provenance edges and, transitively, everything downstream of them. It reads and writes the experiment database, so every subcommand is a post-bootstrap builtin: it is NOT marked knit_usable_before_bootstrap (the central runtime guard refuses it until the experiment is bootstrapped, which is also what makes the --when selector constraints legal), it declares no table with knit_with_table (remove must never record a row it is meant to be deleting), and it runs knit_without_provenance so it emits no call edge of its own.

This file currently provides the command surface, selection resolution, both closure modes, and the erase-set mapping: registration, the shared selector/flag declarations, the resolvers that turn a selector into the set of starting row ids, the fixed-point downward closure over the provenance graph, the callee/artifact refusal check that rejects erasing a callee (or artifact) whose caller (or producer) is kept, the whole-lineage closure that --from-root selects (the connected component over call/produced edges, traversed both directions, with no refusal check), the mapping that resolves every erase-set id to its (table, kind) and reads each artifact's on-disk path and type before the row is gone, the collector that finds the plain (non-artifact) file/directory outputs remove leaves on disk, the non-terminal-job refusal that rejects erasing a job that has not finished, the report builder and printer that render the itemized plan (the data rows, the provenance edges, the on-disk directories and artifact entries to remove, and what is left on disk), and bodies that validate the exactly-one-selector contract, resolve the selection, compute whichever closure the flags request, run the refusal check in the default mode, map the erase set, refuse a non-terminal job, and either print the report and stop under --dry-run, print the report and delete the rows and files under --yes, or print the resulting erase set. The deletion transaction removes, in one atomic BEGIN...COMMIT, every provenance edge touching the erase set and then the data rows table by table. After a successful commit the filesystem phase removes, best effort, each erased job/setup/resource instance directory (setups and resources only when their on-disk marker still names the erased row) and -- unless --keep-artifacts -- each artifact entry under the artifact root, and then reports the files it deliberately left on disk (a command's plain outputs, and kept artifact entries) and exits non-zero listing anything a removal could not clear. --keep-files is stronger: it erases only the database rows and edges and makes no filesystem changes at all (every directory, artifact, and output stays on disk). Unless --yes or --dry-run is given the report is printed and the user is prompted before anything is deleted (a non-interactive stdin without --yes is a fatal refusal), so a removal never happens without either explicit confirmation or --yes.

Declare a remove subcommand's selector parameters, each as an optional string with a --when constraint that enforces mutual exclusion: a selector applies only when every other selector of the same subcommand is empty, so providing two at once is fatal. This makes the exactly-one contract's exclusion half declarative; the presence half (rejecting zero selectors) is a body check (_knit_remove_require_one_selector). Call it between knit_register and knit_done, so the knit_with_optional calls attach to the command being registered.

Parameters:

• kind -- [in] The entity kind, used in the parameter descriptions.

• ... -- [in] The selector names to declare (a subset of id, name, type, group, path).

int _knit_remove_delete_rows()

Delete the erase set from the database in one transaction: first every provenance edge that touches the set, then the data rows table by table. The whole thing is a single ".bail on" BEGIN...COMMIT fed to _knit_sqlite3_write, so it runs under the advisory write lock and either the whole erase set goes or nothing does (a failing statement stops the CLI and rolls the open transaction back). Edges are deleted with "source_id IN (...) OR target_id IN (...)" over every erase-set id, which also clears a KEPT provider's used_by edge into an erased consumer (the edge is caught by its erased target even though its source stays). Data rows are grouped by table from the id -> table map so each table is cleared with one DELETE. Ids are escaped into the IN-lists by _knit_remove_id_in_list, never interpolated raw. Filesystem side effects (job, setup, and resource directories and artifact entries) are handled separately after a successful commit.

Parameters:

• __knit_tables -- [in] Name of the assoc array mapping id -> table.

• ... -- [in] The erase-set ids.

int _knit_remove_dispatch()

Shared body for every remove subcommand: enforce the exactly-one-selector contract, resolve the selection to its starting ids, compute the erase set, map each id to its table and kind, and refuse the operation if it would erase a non-terminal job. Which closure is computed depends on --from-root: without it, the default downward closure is taken and the callee/artifact refusal check runs (fatal before anything is printed or deleted); with it, the whole-lineage connected-component closure is taken and the refusal check is skipped by design. Under --dry-run the itemized report is built and printed and the command stops (no prompt, no deletion). Otherwise the report is printed and, unless --yes was given, the user is prompted to confirm (a non-interactive stdin without --yes is a fatal refusal; an interactive decline stops with exit 0). Once confirmed (or with --yes), the erase set is deleted from the database (edges and rows, in one transaction), the framework-managed on-disk state is removed best-effort (job/setup/resource directories and, unless --keep-artifacts, artifact entries; --keep-files removes nothing on disk), and the files left on disk are reported; the command exits non-zero if any attempted removal could not be cleared. --dry-run wins if both flags are given. The selector names are given as leading arguments up to a literal "--", after which come the command invocation arguments.

Parameters:

• kind -- [in] The entity kind of the subcommand.

• ... -- [in] The selector names, then "--", then the invocation arguments.

int _knit_remove_filesystem()

Remove the framework-managed on-disk state of the erase set, best-effort, after the deletion transaction has committed. For every erase-set id, by kind:

• job: its job directory <job-root>/<id> is removed;

• setup: its instance directory <setup-root>/<name> is removed ONLY when the directory's .setup.id marker names this row -- so erasing an older historical build never removes a newer kept build's live directory (both share the instance name; only one owns the directory);

• resource: its instance directory <resource-root>/<name> and its sibling sidecar markers (.<name>.resource.{type,source,id}) are removed under the same owns-the-dir guard (the .<name>.resource.id sidecar names the row);

• artifact: when keep_mode is "none", its on-disk entry <artifact-root>/<path> is removed (symlink-safe, containment-guarded, empty parents pruned -- see _knit_remove_rm_artifact); "artifacts" keeps it. When keep_mode is "files" this function touches nothing on disk and returns at once: every directory and artifact entry is kept. A command's plain (non-artifact) file/directory outputs are NEVER removed here; the caller lists the surviving ones with _knit_remove_report_left. Each entry whose removal was ATTEMPTED but did not clear is logged and its path is appended to a caller-named failures array, so the caller can print a final "remove by

  hand" list and exit non-zero; a deliberately kept file (a plain output, or an artifact under a keep_mode that spares it) is not a failure. The setup/resource instance names come from the map _knit_remove_instance_names read before deletion, and the artifact paths from the map _knit_remove_map_ids read before deletion.

Parameters:

• __knit_ret -- [out] Name of the array to append unremovable paths to.

• __knit_kinds -- [in] Name of the assoc array mapping id -> kind.

• __knit_apaths -- [in] Name of the assoc array mapping artifact id -> path.

• __knit_names -- [in] Name of the assoc array mapping id -> instance name.

• keep_mode -- [in] Removal policy: "none" removes directories and artifact entries, "artifacts" keeps artifact entries, "files" touches nothing on disk.

• ... -- [in] The erase-set ids.

int _knit_remove_id_in_list()

Build a SQL IN-list -- a comma-separated sequence of single-quoted, escaped ids such as 'a','b','c' -- from a set of ids, returned through a caller-named variable (empty when no non-empty id is given). Every id is escaped with _knit_sql_escape, never interpolated raw. The report builder uses it to select the provenance edges touching the erase set; the deletion phase reuses it.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the IN-list text.

• ... -- [in] The ids.

int _knit_remove_id_table()

Find which table a row id lives in, returned through a caller-named variable, by probing every registered table for the id. Row ids are globally unique across tables, so the first hit is authoritative. A table that was registered but never created is probed harmlessly (the error is silenced). The empty string means the id was not found in any table.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the table name (empty if none).

• id -- [in] The row id to locate.

int _knit_remove_instance_names()

Read the instance name of every setup and resource row in the erase set into a caller-named associative array (id -> name), from the "name" column each such row records. It must run BEFORE the rows are deleted, because the filesystem phase needs the names to locate the instance directories (setups/<name>, resources/<name>) after the rows are gone, exactly as the artifact path is read before deletion. Ids of other kinds are skipped (a job directory is named by its id, an artifact by its path), as is any row whose name column is empty.

Parameters:

• __knit_ret -- [out] Name of the assoc array to fill (id -> instance name).

• __knit_kinds -- [in] Name of the assoc array mapping id -> kind.

• __knit_tables -- [in] Name of the assoc array mapping id -> table.

• ... -- [in] The erase-set ids.

int _knit_remove_map_ids()

Map every erase-set id to the table it lives in and the entity kind of that table, returned through two caller-named associative arrays (id -> table and id -> kind). For each id whose kind is "artifact" it additionally reads that row's on-disk path and type into two more caller-named associative arrays (id -> path and id -> type), because the filesystem phase needs them after the row itself has been deleted. An id whose table cannot be located (its owning command is not registered in the current script, so its table is absent from the registry _knit_remove_id_table probes) is skipped: it cannot be classified or grouped for deletion here. Row ids are globally unique across tables, so the id -> table map doubles as the group-by-table index the delete phase needs.

Parameters:

• __knit_ret1 -- [out] Name of the assoc array to fill (id -> table).

• __knit_ret2 -- [out] Name of the assoc array to fill (id -> kind).

• __knit_ret3 -- [out] Name of the assoc array to fill (artifact id -> path).

• __knit_ret4 -- [out] Name of the assoc array to fill (artifact id -> type).

• ... -- [in] The erase-set ids.

int _knit_remove_plain_outputs()

Collect the plain (non-artifact) file/directory OUTPUTS of every erased row into a caller-named associative array (recorded path -> owning command name), reading the id -> table map knit_remove_map_ids produced. A plain output is a file/directory output a command declared that is NOT an artifact: for each row, the owning command's _KNIT_CMD<cmd>fileparams entries whose marker direction is "output" and whose name is not in _KNIT_CMD<cmd>_artifacts; the recorded path is that row's <name> column. Input file/directory parameters are ignored (they are the run's inputs, not its outputs), and so are artifacts (their on-disk entries are handled by the filesystem phase). The framework tables (jobs/runs/artifacts) carry no user-declared file outputs and are skipped, as is any row whose owning command is no longer registered in the current script -- its file-parameter markers are absent, so its columns cannot be classified. remove never deletes these paths; the caller lists them under "Left on disk".

Parameters:

• __knit_ret -- [out] Name of the assoc array to fill (path -> command).

• __knit_tables -- [in] Name of the assoc array mapping id -> table.

• ... -- [in] The erase-set ids.

int _knit_remove_print_report()

Lay out the itemized report the erase set built. The header line is a parameter so the tense matches the mode ("The following will be permanently erased:" before a prompt or under --dry-run; "Erased:" after the fact under --yes). The data-row section is always printed (the erase set is never empty); the provenance-edge, removed, and "left on disk" sections are printed only when they have entries. Each section header carries its count.

Parameters:

• header -- [in] The header line to print above the report.

• __knit_rows -- [in] Name of the array of data-row lines.

• __knit_edges -- [in] Name of the array of provenance-edge lines.

• __knit_removed -- [in] Name of the array of removed dir/entry paths.

• __knit_left -- [in] Name of the array of "left on disk" lines.

int _knit_remove_report_left()

Print the "NOT removed" report after an actual removal: the files and directories remove deliberately left in place that STILL EXIST. In order:

• when keep_mode is "files", every job/setup/resource directory the removal left on disk, tagged by kind as "(<kind> directory, --keep-files)".

• every plain (non-artifact) file/directory output remove never deletes, attributed to its owning command as "(output of <command>)". A plain output that lived inside a removed job directory is already gone and is not listed.

• when keep_mode is "artifacts" or "files", every kept artifact entry <artifact-root>/<path>, marked "(artifact, <flag>)" where <flag> is the flag that kept it. Only entries that still exist are listed, so this is a truthful record of what survived. When nothing survived, nothing is printed. This report is informational and does not itself set the exit code; a removal that FAILED (as opposed to a deliberately kept file) is handled separately by the caller from the failures _knit_remove_filesystem collects.

Parameters:

• __knit_kinds -- [in] Name of the assoc array mapping id -> kind.

• __knit_apaths -- [in] Name of the assoc array mapping artifact id -> path.

• __knit_plain -- [in] Name of the assoc array mapping plain output path -> command.

• __knit_names -- [in] Name of the assoc array mapping setup/resource id -> name.

• keep_mode -- [in] Removal policy: "artifacts" kept artifact entries; "files" kept every directory and artifact on disk.

• ... -- [in] The erase-set ids.

int _knit_remove_require_one_selector()

Enforce the presence half of the exactly-one-selector contract: at least one of the named selectors must be provided. The mutual-exclusion half (at most one) is enforced declaratively by the --when constraints the selectors carry, so this only refuses the all-empty case. The selector names are given as leading arguments up to a literal "--", after which come the command invocation arguments.

Parameters:

• ... -- [in] The selector names, then "--", then the invocation arguments.

Returns:

Fatal if no selector was provided; otherwise 0.

int _knit_remove_resolve_by_group()

Resolve a --group selector (remove:job) to a starting id set: every job whose "group" column equals the given name (a whole prepare batch). The reserved "group" identifier is quoted. No match is fatal.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting ids.

• group -- [in] The group name to resolve.

int _knit_remove_resolve_by_id()

Resolve a --id selector to a one-element starting id set, verifying the id exists and belongs to the expected kind. A missing id is fatal; an id that exists but in a table of a different kind is fatal with a hint to use the right subcommand (e.g. remove:setup --id given a job id).

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting id.

• kind -- [in] The expected entity kind.

• id -- [in] The row id to resolve.

int _knit_remove_resolve_by_name()

Resolve a --name selector (the instance name given at creation) to a starting id set. For setup/resource the name is scanned across every table of that kind via the "name" column, so it resolves from the database even after the instance directory is gone. For a job the name is the "jobs.name" alias. For run the name is the launched app (the runs "app" column); for app/command the name is the command/app name, which is its table. No match is fatal.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting ids.

• kind -- [in] The entity kind.

• name -- [in] The instance/command name to resolve.

int _knit_remove_resolve_by_path()

Resolve a --path selector (remove:artifact) to a starting id set. The artifacts "path" column is UNIQUE, so at most one row matches. No match is fatal.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting id.

• path -- [in] The artifacts-relative path to resolve.

int _knit_remove_resolve_by_type()

Resolve a --type selector (every instance of a type) to a starting id set. For setup/resource the type is the per-command table (setup:<type> / resource:<type>), so this selects every row in it. For a job the type is the job-body table; the starting ids are the job submissions (jobs rows) whose body rows live in that table, reached by the "call" edge from the submission to its body. No match is fatal.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting ids.

• kind -- [in] The entity kind (setup, resource, or job).

• type -- [in] The type to resolve.

int _knit_remove_resolve_selection()

Resolve a remove subcommand's selection into its starting id set, returned through a caller-named array. Reads whichever selector was provided (the exactly-one contract is already enforced by the --when constraints and the body presence check) and dispatches to the per-selector resolver for the kind. Every resolver is fatal on zero matches; one or many matches all become starting ids.

Parameters:

• __knit_ret -- [out] Name of the array to fill with the starting ids.

• kind -- [in] The entity kind of the subcommand.

• ... -- [in] The command invocation arguments (the selectors are read here).

int _knit_remove_rm_artifact()

Remove one on-disk artifact entry at <root>/<rel> best-effort, with a containment guard and empty-parent pruning. An absent entry is a success. The entry itself is never resolved, so a --link-from artifact (a symlink under the artifact root) is removed as the link and a target outside the root is left untouched; only the entry's PARENT is resolved, and the removal is refused (silently, as a no-op) unless that parent stays inside the artifact root -- a cheap defense, since the recorded path is framework-written and always relative. After a successful removal, now-empty parent directories are pruned up to but not including the artifact root, so the root does not accumulate empty subtrees.

Parameters:

• root -- [in] The resolved artifact root.

• rel -- [in] The artifacts-relative path recorded for the entry.

Returns:

0 if the entry is gone (or was absent, or outside the root); 1 if it remains.

int _knit_remove_rmtree()

Remove a single filesystem entry (a file, directory, or symlink) best-effort. An absent entry is a success (nothing to do). A symlink is removed as the link, never followed to its target (rm -rf never deletes a symlink's target). The return code reports the OUTCOME, not rm's exit status: 0 when the entry is gone afterward (or was never there), 1 when it still exists, so the caller can list a path it could not clear.

Parameters:

• path -- [in] The entry to remove.

Returns:

0 if the entry is gone (or was absent); 1 if it remains.

int _knit_remove_row_value()

Read a single column of a single row, identified by id, into a caller-named variable (empty when the row, the column, or the table is absent -- the error is silenced). The report builder uses it to fetch a display field for one erase-set row: a setup or resource instance name, a job name or state, or a run's launched app.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the column value.

• table -- [in] The table to read from.

• id -- [in] The row id.

• column -- [in] The column name to read.

int _knit_remove_table_kind()

Resolve a table name to the remove entity kind of the rows it holds, returned through a caller-named variable. The framework tables map directly (jobs -> job, runs -> run, artifacts -> artifact); any other table maps through its owning command's KNIT_CMD<cmd>_type marker (setup / resource / app, or command for a plain command). A wrapper table is reported as "command" because remove:command covers wrappers. An unregistered table (no owning command) yields the empty string.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the kind (empty if unknown).

• table -- [in] The table name to classify.

int _knit_remove_tables_of_kind()

Fill a caller-named array with every table that holds rows of a given entity kind. The framework kinds resolve to a single table each (job -> jobs, run -> runs, artifact -> artifacts); the other kinds (setup, resource, app, command) are gathered from the live table registry, skipping the framework tables (which carry their own kinds) and keeping only those whose owning command's kind matches. This is what lets a setup/resource --name scan every table of that kind.

Parameters:

• __knit_ret -- [out] Name of the array to fill with table names.

• kind -- [in] The entity kind to gather tables for.

resource.sh

Functions

int _knit_command_is_resource()

Test whether a command is a resource type, i.e. it was registered with knit_register_resource. Used by the resource declaration directives to reject use on any other kind of command. Reads the command kind from the KNIT_CMD<cmd>_type field (see knit_register).

Parameters:

• cmd -- [in] Command (mangled name) to test.

Returns:

0 if the command is a resource type, 1 otherwise.

int _knit_fetch()

Body of the builtin fetch dispatcher command. It materializes a named resource instance at <resource-root>/<name> by dispatching to the requested resource type, mirroring knit setup:

./exp.sh fetch --name <name> [--ignore-checksum] -- <resource-type> [args...]

The instance name is validated as a single path component and the type must be a registered resource type. Fetching is idempotent by name and serialized on a per-name lock under .knit: if the instance already exists with a matching source identity the fetch is a no-op that just reprints the path; if it exists with a different source it is fatal (remove it to re-fetch); otherwise the download body runs, the instance is recorded, and its type / source identity / row id are written to sidecar markers beside it. The resolved instance path is printed to stdout (all logging is on stderr). On a failed download the partial instance is removed and no row survives. When the type declares knit_with_checksum and the pin still applies (the source defaults are unchanged), it is exported as KNIT_RESOURCE_EXPECTED_CHECKSUM for the body to verify; --ignore-checksum (exported as KNIT_IGNORE_CHECKSUM) bypasses that verification.

int _knit_fetch_git()

Git backend: clone <url> into <dest>, check out <ref>, and print the resolved commit SHA (git rev-parse HEAD) to stdout so a mutable ref is pinned to the commit actually obtained. The clone and checkout write their progress to stderr (redirected here so stdout carries only the SHA), and the resulting tree is made read-only. Returns non-zero (leaving cleanup to the caller) if git is missing or any step fails.

Parameters:

• dest -- [in] Directory to create and clone into (must not already exist).

• url -- [in] Repository URL to clone.

• ref -- [in] Ref (branch, tag, or commit) to check out.

int _knit_fetch_local()

Local backend: materialize <dest> from a local source <src>. By default <dest> is a symlink to <src> (so a large already-staged dataset is not duplicated), left writable through its target. With <copy> = "true" the source is copied into a self-contained snapshot which is then made read-only. The source path is resolved to an absolute path first, so a symlink instance does not depend on the caller's working directory. When <expected> is a non-empty knit_with_checksum pin the instance's sha256 (of the file, or a recursive digest of a directory) is checked against it and a mismatch fails the fetch. Returns non-zero (leaving cleanup to the caller) if the source is missing or a step fails.

Parameters:

• dest -- [in] Path to create for the instance (must not already exist).

• src -- [in] Local source path (file or directory) to link or copy.

• copy -- [in] "true" to copy a read-only snapshot, "false" to symlink the source.

• expected -- [in] Expected sha256 to verify, or "" to skip verification.

int _knit_fetch_url()

URL backend: download <url> with curl into <dest>, optionally unpacking it. The artifact is saved under its URL basename; when <uncompress> is "true" it is extracted with tar -xf (auto-detecting gzip/bzip2/xz) and the archive is then removed, so the instance holds the unpacked tree. curl uses -f so an HTTP error fails the fetch. When <expected> is a non-empty knit_with_checksum pin the downloaded archive's sha256 is checked against it (before it is unpacked or removed) and a mismatch fails the fetch. The resulting instance is made read-only. Returns non-zero (leaving cleanup to the caller) on any failed step.

Parameters:

• dest -- [in] Directory to create and download into (must not already exist).

• url -- [in] URL of the artifact to download.

• uncompress -- [in] "true" to unpack the downloaded archive, "false" to keep it.

• expected -- [in] Expected archive sha256 to verify, or "" to skip verification.

int _knit_resource_check_method()

Done callback installed by knit_register_resource. Fatal if the resource type completed registration without declaring a download method, so a type must declare exactly one of knit_with_git / knit_with_url / knit_with_local.

Parameters:

• cmd -- [in] The resource type's command (mangled name).

int _knit_resource_check_sha()

Compare a computed digest against an expected knit_with_checksum pin, case- insensitively (sha256 hex is case-independent). On a mismatch it prints a knit_error naming what failed and returns 1, so the caller aborts the fetch and removes the partial instance; on a match it returns 0.

Parameters:

• expected -- [in] The expected value (the knit_with_checksum pin).

• actual -- [in] The computed value.

• what -- [in] Short label for the artifact (for the error message).

int _knit_resource_cleanup_dir()

Remove a partial or failed instance at <dir>, restoring write permission first so a read-only tree (see _knit_resource_make_readonly) can be deleted. A symlink instance is unlinked without touching its target. A no-op when <dir> is empty or does not exist, so it is safe to call unconditionally on any failure path.

Parameters:

• dir -- [in] Path to the instance to remove.

int _knit_resource_defaults_used()

Test whether a fetch still uses the resource type's declared source defaults, so a knit_with_checksum pin still applies. The pin is authored against the default source; if the user overrides the source the artifact would legitimately hash differently, so the pin no longer holds. For each source-defining parameter of the backend (git: url + ref; url: url; local: path) it compares the effective value against the declared default; it returns 0 when every one matches (defaults in use), 1 otherwise. The uncompress/copy flags are not compared: they do not change the archive/file content the pin covers.

Parameters:

• cmd -- [in] The resource type's command (mangled name).

• method -- [in] The backend method ("git", "url", or "local").

• ... -- [in] The fetch's expanded arguments (read via knit_get_parameter).

int _knit_resource_dep_after_cb()

After-callback installed by knit_with_resource on the consuming command, one per declared resource parameter. It records a "used_by" edge from the fetched resource instance the command depends on to the command itself. It runs as an after-callback (not the before-callback that validates the instance) because the consumer's frame is on the executing stacks only from push time onward, so the consumer's resolved row id — the edge target — is available here but not in the before-callback (this mirrors _knit_setup_dep_after_cb).

The declared parameter name and required type are bound at registration time and carried as the first two arguments; the trailing arguments are the command's own runtime arguments, scanned for the instance name. A missing name is silently skipped: the before-callback already fataled on it, so this is only reached with a valid instance.

Parameters:

• param -- [in] Normalized name of the resource parameter to read.

• type -- [in] Resource type of the named instance (unused; kept for symmetry with the before-callback's bound arguments).

int _knit_resource_dep_before_cb()

Before-callback installed by knit_with_resource on the consuming command, one per declared resource parameter. Before the command body runs it validates that the resource named by the parameter has been fetched and is of the required type, using only the on-disk sidecar markers (no database read):

• the parameter value (the instance name) missing → fatal;

• <resource-root>/<name> absent → fatal, printing the knit fetch command to run;

• the .<name>.resource.type sidecar marker absent or not equal to the declared type → fatal type-mismatch. The instance name is also validated as a single path component. Returning normally lets the command proceed; a fatal aborts it before the body runs.

The declared parameter name and required type are bound at registration time and carried as the first two arguments; the trailing arguments are the command's own runtime arguments, scanned for the parameter value.

Parameters:

• param -- [in] Normalized name of the resource parameter to read.

• type -- [in] Resource type the named instance must have been fetched as.

int _knit_resource_fetch_body()

Shared body registered for every resource type (the function every fetch:<type> command runs). It reads the type's declared download method from the per-command marker (KNIT_CMD<cmd>_fetch_method) and dispatches to the matching backend, materializing the instance at KNIT_RESOURCE_PREFIX (the target path the knit fetch dispatcher exports; it must not already exist, since each backend creates it). The git backend's resolved commit SHA is recorded as the "commit" output. On any backend failure the partial instance is removed and the body returns non-zero so the dispatcher aborts without recording a row.

When the type declares knit_with_checksum and the pin still applies (the source defaults are unchanged), the dispatcher exports it as KNIT_RESOURCE_EXPECTED_- CHECKSUM; the body verifies it — the git commit SHA directly, and the url/local artifacts through their backends — unless the fetch bypassed it with --ignore-checksum (KNIT_IGNORE_CHECKSUM). A mismatch fails the fetch.

int _knit_resource_make_readonly()

Make a fetched instance tree read-only so consumers cannot mutate a shared input: chmod -R a-w clears the write bit for everyone while preserving the existing read and execute bits. Applied by the git and url backends and by the local backend's --copy path; a symlinked local instance is deliberately left alone (its target stays owned and writable by whoever staged it).

Parameters:

• dir -- [in] Path to the instance tree (or file) to make read-only.

int _knit_resource_param_type()

Store the resource type a parameter was declared with in the caller-named variable, or the empty string when the parameter is an ordinary parameter (not declared through knit_with_resource). Reads the per-parameter marker (KNIT_CMD<cmd>resource) that knit_with_resource records, so describe and --help can annotate a resource parameter with its type from the registration tables alone (no database read). The parameter name must be normalized, as it is stored in the parameter sets.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resource type (empty if none).

• cmd -- [in] Mangled command name.

• param -- [in] Normalized parameter name.

int _knit_resource_record_used_by_edge()

Record a "used_by" provenance edge from a fetched resource instance to a consuming invocation. The edge's source is the instance (its row id read from the .<name>.resource.id sidecar, its node name "fetch:<type>" -- the owning command that recorded the row, so the source_name matches the instance's own "call"/"executed" edges and the query transpiler's label resolution -- with the type read from the .<name>.resource.type sidecar); its target is the consumer. Delegates the gated write to _knit_record_used_by_edge, so it records nothing when recording is disabled, on a suppressed rank, before bootstrap, when the target does not participate in the graph, or when the sidecar has no id (e.g. an instance fetched before provenance shipped).

Parameters:

• name -- [in] Resource instance name (as passed to knit fetch --name).

• target_cmd -- [in] Mangled command name of the consumer (the edge target).

• target_id -- [in] Resolved row id of the consumer (the edge target).

int _knit_resource_require_registration()

Fatal unless a resource type is currently being registered (i.e. this is called between knit_register_resource and knit_done). Shared by the download decorators and knit_with_checksum, which are valid only on a resource type.

Parameters:

• directive -- [in] Name of the calling directive (for the error message).

int _knit_resource_root()

Store the resolved resource root — the directory under which resource instances live — in the caller-named variable. Reads the verbatim resource_path from the metadata table (falling back to "resources" when unset, for robustness) and resolves it against the experiment root via _knit_resolve_experiment_path. Mirrors _knit_setup_root / _knit_job_root.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved resource root.

int _knit_resource_set_method()

Record the download method chosen by a download decorator in the current resource type's per-command marker (KNIT_CMD<cmd>fetch_method), enforcing the "exactly one download decorator per resource type" rule: a second call (whether the same decorator twice or two different backends) is fatal. Mirrors the _KNIT_CMD<cmd>_spack_env at-most-once marker pattern.

Parameters:

• method -- [in] The backend name ("git", "url", or "local").

int _knit_resource_source_identity()

Build the source-identity string for a fetch and store it in the caller-named variable. The identity is the tuple that identifies where an instance came from, recorded on its row and compared on a same-name re-fetch to distinguish an idempotent skip (identical source) from a conflicting re-fetch (same name, a different source). It is derived from the backend method and the resolved parameters, per backend:

• git: the repository url and the requested ref

• url: the url and the uncompress flag

• local: the source path and the copy flag The commit a git ref resolves to is deliberately not part of the identity: a mutable ref (e.g. main) still identifies the same source between fetches.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the identity string.

• method -- [in] The backend method ("git", "url", or "local").

• ... -- [in] The fetch's arguments (read via knit_get_parameter).

int _knit_sha256()

Compute the sha256 of a file or directory and store the 64-hex digest (with no trailing filename and no algorithm prefix) in the caller-named variable. A regular file is hashed directly. A directory is hashed recursively into one digest: every regular file under it is hashed together with its relative path (sha256sum prints "<hash> <path>"), the lines are sorted for a stable order independent of readdir order, and the aggregate is hashed again — so both the tree structure and every file's content contribute. A symlink is followed (so a local symlink instance hashes its target). Returns non-zero (with a knit_error) when sha256sum is unavailable or a step fails.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the digest.

• path -- [in] Path to the file or directory to hash.

Variables

AssociativeArray _KNIT_RESOURCES

Associative array mapping registered resource type names to 1. Populated by knit_register_resource and consulted by the knit fetch dispatcher (to resolve a requested type) and by knit_with_resource (to validate a declared dependency's type). Declared global (-g) so it survives being sourced from within a function (as the bats tests do). Mirrors _KNIT_SETUPS.

rocrate.sh

Functions

int _knit_bundle_rocrate_generate()

Write the RO-Crate manifest (ro-crate-metadata.json) for the experiment. The manifest is JSON-LD in the RO-Crate 1.1 / Process Run Crate vocabulary: a metadata descriptor, a root Dataset that conforms to the profile and lists the packed files (hasPart) and the recorded actions (mentions), a File or Dataset data entity per packed path, a SoftwareApplication per distinct command, one CreateAction per provenance node, and the PropertyValue entities for the rows' columns. The action graph comes from the provenance edges: a "call" edge links the caller's result to the callee action; a "used_by" edge links the consumer's object to the setup or resource action; a "produced" edge links the producer's result to the artifact. Row columns become PropertyValues, split into the action's object (inputs) and result (outputs); "native_cmd" becomes the action's description and "state" its actionStatus.

The manifest describes exactly the given packed paths, so the same generator serves "knit bundle --ro-crate" (the files travel with it) and "knit export

ro-crate" (the manifest alone, describing the files in place).

Parameters:

• output -- [in] The path to write, or "-" for standard output.

• root -- [in] The absolute experiment root.

• ... -- [in] The packed relative paths the manifest must describe.

int _knit_export_rocrate()

Body of "knit export ro-crate": write the RO-Crate manifest alone, with no archive and no copied files. It describes the experiment's on-disk files by their current relative paths, using the same default contents "knit bundle" would pack, so it is the cheap way to inspect or regenerate the manifest in place. The command is read-only: it declares no table and takes knit_without_provenance.

Parameters:

• ... -- [in] The command invocation arguments.

int _knit_rocrate_edges_json()

Print the provenance edges, as a JSON array of objects, one per edge, read straight from the provenance table with sqlite's JSON output. The bootstrap subtree is filtered out (an edge naming "bootstrap" at either end), so the crate describes the science, not the plumbing. The empty array is printed when the table is absent or holds no non-bootstrap edge.

int _knit_rocrate_encoding_format()

Print a best-guess IANA media type for a packed file, chosen from its base name, or nothing when no guess fits. The RO-Crate manifest records the type as a File entity's encodingFormat, so a reader knows how to open the file without Knit. The guesses cover the file kinds a bundle carries: the provenance database, the experiment and job scripts, the JSON lock files, the Spack YAML manifests, and the plain-text logs and markers.

Parameters:

• name -- [in] The file's base name.

int _knit_rocrate_files_json()

Print, as a JSON array of objects, one entry per packed file, describing it for the RO-Crate data entities. Each entry holds the packed relative path, the @id it takes in the crate (a trailing "/" for a directory), the base name, a directory flag, a guessed encodingFormat, whether it is the experiment script, and — for a file under a job directory — the job's id (so an action can list its own logs as results). The manifest file itself is never described as a data entity, so it is skipped here.

Parameters:

• root -- [in] The absolute experiment root.

• script_rel -- [in] The experiment script's path relative to the root.

• job_rel -- [in] The job root's path relative to the root ("jobs" by default).

• ... -- [in] The packed relative paths.

int _knit_rocrate_outputs_json()

Print, as a JSON array of strings, the normalized output column names of a command. The RO-Crate mapping puts a command's inputs (params and flags) in an action's "object" and its outputs in the action's "result", so the generator needs to know which of a data row's columns are outputs. The names come from the command's registration output set and are normalized (hyphens to underscores) to match the database column names. An unknown command, or one with no outputs, yields the empty array.

Parameters:

• mangled -- [in] The mangled command name.

int _knit_rocrate_rows_json()

Print the recorded data rows, as a JSON array of objects, one per row, drawn from every per-command table that is both registered in this run and present in the database. Each object carries the row id, its table and owning command, the command's output column names, and the row's full column map ("cols"), so the generator can turn a row into an action's PropertyValues and split them into inputs and outputs. A table with no rows contributes nothing.

int _knit_rocrate_table_exists()

Test whether a table exists in the database. Used before a "SELECT *" read so a per-command table that was registered but never written (no invocation yet) does not draw an error.

Parameters:

• table -- [in] The table name.

Returns:

0 if the table exists, 1 otherwise.

sched.sh

Functions

int _knit_sched_backend()

Resolve which scheduler backend to use: bootstrap metadata (scheduler), else live detection. Detection's "<unknown>" (no batch scheduler present) means the workstation case and maps to the local background-process backend. An explicit "none" in metadata is a real, deliberate backend (a user-owned cluster driven without a scheduler) and flows through untouched. Returns one of "local", "none", "slurm", "pbs", "flux".

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved backend name.

int _knit_sched_cancel()

Dispatch to the configured backend's cancel function, which asks the scheduler to terminate a running job. Each backend uses its native primitive (local: kill, slurm: scancel, pbs: qdel). Cancelling a job that is already gone is not an error. Knit's terminal state (killed) is recorded by the caller.

Parameters:

• backend -- [in] Scheduler backend name ("local", "none", "slurm", "pbs", "flux").

• jobid -- [in] Backend job id (scheduler id, or a PID for local/none backends).

int _knit_sched_directives()

Dispatch to the configured backend's directive generator, printing the batch scheduler directive lines (e.g. "#SBATCH ..." / "#PBS ...") for the resolved options. The local and none backends print nothing.

Parameters:

• backend -- [in] Scheduler backend name ("local", "none", "slurm", "pbs", "flux").

• arr_name -- [in] Name of the resolved-options associative array.

• jobdir -- [in] Job directory (used by backends for --output/--error paths).

int _knit_sched_hostfile()

Print, one per line, the raw host entries a job is running on, as the current backend's scheduler reports them. Unlike the other dispatchers this resolves the backend itself (via _knit_sched_backend): it is meant to be called at runtime from inside a job body, where no backend has been resolved by a caller.

The output is the "raw" host list: hostnames may be repeated (once per slot, e.g. PBS's $PBS_NODEFILE) or carry trailing ":N" info. Callers that want a deduplicated, cleaned list post-process it (see knit_job_hostnames).

int _knit_sched_profile_field()

Return a field from the bootstrapped experiment's machine profile, or the empty string when no profile is configured. The value comes from the profile JSON frozen at bootstrap (profile_json metadata), read via knit_get_profile_field. The profile argument is retained as a gate so callers can request a field unconditionally.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the field value.

• profile -- [in] Profile label (may be empty; empty means "no profile").

• jq_path -- [in] jq path expression, e.g. '.scheduler.default_queue'.

int _knit_sched_resolve()

Resolve the submission options for a job into a caller-provided associative array. For each option the value is resolved by precedence:

explicit CLI argument -> bootstrap metadata -> machine profile -> hard-coded

The array is keyed by canonical option name (job-name, account, project, queue, nodes, cpus-per-node, walltime, gpus-per-node, wait) plus "extra-args" for the site-mandatory scheduler arguments captured at bootstrap, and "walltime-defaulted" ("true"/"false") recording whether the walltime was chosen by knit rather than requested (so the submit path can warn). The per-node core count (cpus-per-node) is derived rather than requested (knit allocates whole nodes). Job stdout/stderr are fixed to <job-dir>/.stdout and <job-dir>/.stderr by the backend, so they are not resolved here.

Parameters:

• out_array -- [out] Name of an associative array to populate (passed by name).

• ... -- [in] The submission CLI arguments (everything before "--").

int _knit_sched_submit()

Dispatch to the configured backend's submission function, which submits the already-written batch script and prints the resulting scheduler job id (or, for the local/none backend, the process id) to stdout.

Parameters:

• backend -- [in] Scheduler backend name ("local", "none", "slurm", "pbs", "flux").

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

• jobdir -- [in] Job directory (holds .stdout/.stderr for local/none backends).

int _knit_sched_submit_cmdline()

Dispatch to the configured backend's submission-command builder, which fills a caller-provided array (passed by name) with the argv the backend runs to submit the batch script (e.g. "sbatch <script>", "qsub <script>", "bash <script>"). This is built separately from _knit_sched_submit so the resolved submission command can be recorded in the jobs table and logged before it is issued.

Parameters:

• backend -- [in] Scheduler backend name ("local", "none", "slurm", "pbs", "flux").

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

• argv_name -- [in] Name of the array to fill with the submission argv.

int _knit_sched_wait()

Dispatch to the configured backend's blocking wait, which returns once the scheduler no longer considers the job active. Each backend blocks using the native mechanism its scheduler actually provides (see the per-backend functions); knit's own terminal state (completed/killed) is read from the jobs table afterwards, so this only has to unblock when the job stops running.

Parameters:

• backend -- [in] Scheduler backend name ("local", "none", "slurm", "pbs", "flux").

• jobid -- [in] Backend job id (scheduler id, or a PID for local/none backends).

int _knit_sched_write_jobscript()

Write the batch script that a scheduler runs on the compute node. The script carries the backend's directives, exports the job/setup prefixes, cd's into the job directory, and re-enters the experiment script to run the job body via exp.sh submit <job-name> <args>. Arguments are q-quoted so they survive the round-trip through the batch script unchanged.

Parameters:

• script_path -- [in] Path of the batch script to create.

• backend -- [in] Scheduler backend name.

• arr_name -- [in] Name of the resolved-options associative array.

• setup_path -- [in] Setup directory (exported as KNIT_SETUP_PREFIX; empty for a setup-less job, in which case KNIT_SETUP_PREFIX is not exported).

• jobdir -- [in] Job directory (exported as KNIT_JOB_PREFIX; the cd target).

• source_id -- [in] Submitting invocation's row id (exported as KNIT_SOURCE_ID so the compute-side job body records a call edge back to the submitter; empty to omit the export).

• source_command -- [in] Submitting invocation's command name (exported as KNIT_SOURCE_COMMAND; emitted only when source_id is set).

• job_name -- [in] Registered job name to run.

• ... -- [in] Arguments to pass to the job.

int _knit_uuidv7()

Generate a version-7 UUID (RFC 9562) and print it to stdout.

A uuidv7 encodes a 48-bit big-endian Unix millisecond timestamp in its leading bits, so lexically sorting uuidv7 strings orders them by creation time. This is why job directories are named with one: they sort chronologically.

Layout (hex digits, formatted 8-4-4-4-12):

• digits 1-12 : 48-bit millisecond timestamp

• digit 13 : version nibble, always "7"

• digits 14-16 : random

• digit 17 : variant nibble, one of 8, 9, a, b

• digits 18-32 : random

Randomness comes from /dev/urandom when available, falling back to the bash ${RANDOM} generator; the timestamp prefix guarantees ordering and near uniqueness even in the fallback case. The millisecond clock falls back to whole-second precision when date lacks nanosecond (N) support.

Variables

String _KNIT_SCHED_KILL_WARNING_SEC

Seconds before the walltime limit at which a scheduler is asked to warn the job (Slurm --signal). The warning lets the job record itself as "killed" before it is hard-killed (see _knit_job_killed_trap).

String _KNIT_SCHED_POLL_INTERVAL

Seconds between polls when a backend has to wait for a job by polling its scheduler (Slurm squeue, PBS qstat — neither offers a reliable blocking "wait for completion" primitive). Overridable, chiefly so tests can drive the poll loops quickly.

sched_flux.sh

Functions

int _knit_sched_flux_cancel()

Cancel a Flux job with flux cancel. Flux sends SIGTERM then SIGKILL after a grace period, which lets the batch shell's pre-termination handler record the job "killed" (see _knit_job_killed_trap). Cancelling an already-finished or unknown job may print a diagnostic; that is not treated as a knit-level failure since the job is gone.

Parameters:

• jobid -- [in] Flux job id (from the job's .job.id).

int _knit_sched_flux_directives()

Emit the "# flux:" directive lines for a resolved job. A Flux batch directive is a comment whose body starts with "flux:", the analogue of "#SBATCH"/"#PBS". Knit allocates whole nodes exclusively, so --exclusive is always set with --nodes. Job stdout/stderr are fixed to <jobdir>/.stdout and <jobdir>/.stderr. The per-node core count is not emitted: the allocation is whole-node exclusive, so per-node task sizing belongs to the launcher (flux run), not the allocation. Optional fields (bank, queue, gpus) are emitted only when set.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• jobdir -- [in] Job directory (for the --output/--error paths).

int _knit_sched_flux_hostfile()

Print the host list for a Flux job, one hostname per line. flux hostlist expands a job's hostlist; the source "local" reads the enclosing job from $FLUX_JOB_ID, and "-ed '\n'" expands every host with a newline delimiter. This is a per-node list. If the call fails (e.g. not running inside a Flux job), warn and fall back to this machine's hostname, exactly as the other backends do.

int _knit_sched_flux_parse_jobid()

Reduce flux batch output to the bare job id by keeping the first line and trimming surrounding whitespace. The id is a single opaque FLUID token (e.g. "f2QzoR8xF"), so no further splitting is done.

Parameters:

• raw -- [in] Raw stdout captured from flux batch.

int _knit_sched_flux_submit()

Submit an already-written batch script with flux batch and print the resulting job id. Flux has no sbatch --wait equivalent at submit time, so when the resolved "wait" flag is "true" the function blocks after submission with flux job status <id> (a real blocking wait; see _knit_sched_flux_wait). The --output/--error redirection is carried by the script's directives, so the job directory is not needed here.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

• jobdir -- [in] Job directory (unused; redirection is set via directives).

int _knit_sched_flux_submit_cmdline()

Build the flux batch submission command into a caller-provided array, passed by name: "flux batch <script>". Flux has no submit-time blocking flag (see _knit_sched_flux_submit for how the wait option is honoured), so the argv does not depend on the resolved "wait" option.

Parameters:

• argv_name -- [out] Name of the array to fill with the submission argv.

• arr_name -- [in] Name of the resolved-options associative array (unused).

• script -- [in] Path to the batch script to submit.

int _knit_sched_flux_wait()

Block until the Flux job completes. flux job status <id> blocks until the job reaches a terminal state and exits with the job's largest task exit code. It works for any job (it does not need the "waitable" flag), so unlike the Slurm/PBS backends this is a single blocking call, not a poll loop. The job's knit terminal state is read from the DB by the caller afterwards.

Parameters:

• jobid -- [in] Flux job id (from the job's .job.id).

int _knit_sched_flux_walltime_fsd()

Convert an HH:MM:SS walltime to a Flux Standard Duration (FSD) in whole seconds with a trailing "s", e.g. "01:00:00" -> "3600s". Flux --time-limit reads a bare number as minutes, so the explicit "s" suffix avoids that ambiguity. A value that is not HH:MM:SS is printed verbatim so a site that already writes a valid FSD (e.g. "30m") is passed through unchanged.

Parameters:

• walltime -- [in] Walltime string, normally "HH:MM:SS".

sched_local.sh

Functions

int _knit_sched_local_cancel()

Cancel a locally-launched job by sending SIGTERM to its process. SIGTERM (not SIGKILL) is used deliberately: the running job installs a handler for it (see _knit_job_killed_trap) so it can record itself "killed" before exiting. A pid that is not a positive integer, or a process that is already gone, is treated as nothing to do.

Parameters:

• pid -- [in] Process id recorded in the job's .job.id by the local backend.

int _knit_sched_local_directives()

Emit the batch directives for the local backend. The local backend runs the job as a background process with no scheduler, so there are no directives and this prints nothing. It exists so every backend honours the same contract.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array (unused).

• jobdir -- [in] Job directory (unused).

int _knit_sched_local_hostfile()

Print the host list for the local backend. A local run has no scheduler and no node allocation, so it runs on a single host: print this machine's hostname.

int _knit_sched_local_submit()

Submit a batch script on a machine with no scheduler by running it as a detached background process via _knit_submit_local. Job stdout/stderr are redirected to <jobdir>/.stdout and <jobdir>/.stderr, and the resolved walltime (if any) caps the run. When the resolved "wait" flag is "true", block until the process finishes. Prints the process id to stdout.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to run.

• jobdir -- [in] Job directory holding .stdout/.stderr.

int _knit_sched_local_submit_cmdline()

Build the local backend's submission command into a caller-provided array, passed by name: "bash <script>". The local backend runs this in the background via _knit_submit_local, which adds stdout/stderr redirection and an optional walltime cap; those are knit-managed conveniences rather than part of the job command, so the recorded/traced command is the bare "bash <script>".

Parameters:

• argv_name -- [out] Name of the array to fill with the submission argv.

• arr_name -- [in] Name of the resolved-options associative array (unused).

• script -- [in] Path to the batch script to run.

int _knit_sched_local_wait()

Block until a locally-launched job process exits. The job wait command runs in a different process than the submit that launched the job, so the pid is not a child of this shell and the wait builtin cannot be used; instead poll kill -0 (a liveness probe that sends no signal) every _KNIT_SCHED_POLL_INTERVAL seconds until the process is gone. A pid that is not a positive integer, or is already gone, returns immediately.

Parameters:

• pid -- [in] Process id recorded in the job's .job.id by the local backend.

sched_none.sh

Functions

int _knit_sched_none_cancel()

Cancel a none-backend job by signalling its process, identically to the local backend. Delegates to _knit_sched_local_cancel.

Parameters:

• pid -- [in] Process id recorded in the job's .job.id.

int _knit_sched_none_directives()

The "none" backend is for a user who owns a multi-node cluster but does not submit through a batch scheduler (SSH-reachable nodes, a hand-maintained MPI hostfile, etc.). A job launches exactly as the local backend does — as a background process on the submitting host — so the lifecycle functions (directives, submit, cancel, wait) delegate to their local counterparts. The only difference is host reporting: the allocation is the node list configured at bootstrap via --default-nodefile (metadata default_nodefile), not just the submitting host's name. That is what _knit_sched_none_hostfile provides.

Emit the batch directives for the none backend. Like the local backend, the job runs as a background process with no scheduler, so there are no directives. Delegates to _knit_sched_local_directives.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• jobdir -- [in] Job directory.

int _knit_sched_none_hostfile()

Print the host list for a none-backend job: the contents of the node file configured at bootstrap (metadata default_nodefile), one host per line. This is the "raw" host list per the dispatcher contract — entries may repeat or carry a trailing ":N" slot count; callers such as knit_job_hostnames deduplicate and clean it. Blank lines are dropped so an empty entry never reaches the normaliser. When no readable node file is configured (unset, or the path does not exist / is not readable), warn and fall back to this machine's hostname.

int _knit_sched_none_submit()

Submit a batch script for the none backend by running it as a detached background process on the submitting host, identically to the local backend. Delegates to _knit_sched_local_submit, which handles stdout/stderr redirection, walltime, and optional blocking, and prints the process id.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to run.

• jobdir -- [in] Job directory holding .stdout/.stderr.

int _knit_sched_none_submit_cmdline()

Build the none backend's submission command, identically to the local backend (the job runs as a background "bash <script>" on this host). Delegates to _knit_sched_local_submit_cmdline.

Parameters:

• argv_name -- [in] Name of the array to fill with the submission argv.

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to run.

int _knit_sched_none_wait()

Block until a none-backend job process exits. Identical to the local backend (the job is a background process on this host). Delegates to _knit_sched_local_wait.

Parameters:

• pid -- [in] Process id recorded in the job's .job.id.

sched_pbs.sh

Functions

int _knit_sched_pbs_cancel()

Cancel a PBS job with qdel. qdel sends SIGTERM then SIGKILL, letting the batch shell's pre-termination handler record the job "killed" (see _knit_job_killed_trap). A qdel of an already-finished/unknown job may print a diagnostic; that is not treated as a knit-level failure since the job is gone.

Parameters:

• jobid -- [in] PBS job id (from the job's .job.id).

int _knit_sched_pbs_directives()

Emit the #PBS directive lines for a resolved job. Knit allocates whole nodes exclusively: the resource request is one chunk per node (-l select=<nodes>) with -l place=excl. When the per-node core count is known it pins ncpus and mpiprocs on the chunk (mpiprocs is what populates $PBS_NODEFILE, i.e. the launchable slots); otherwise the chunk takes the site defaults. Job stdout/stderr are fixed to <jobdir>/.stdout and <jobdir>/.stderr. Optional fields (account, project, queue, gpus) are emitted only when set.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• jobdir -- [in] Job directory (for the -o/-e paths).

int _knit_sched_pbs_hostfile()

Print the host list for a PBS job. PBS writes the allocated slots to the file named by $PBS_NODEFILE, one hostname per line, each repeated once per launchable slot (mpiprocs) on that node. Print its contents verbatim. If $PBS_NODEFILE is unset or unreadable (e.g. not running inside a PBS job), warn and fall back to this machine's hostname.

int _knit_sched_pbs_parse_jobid()

Reduce a qsub job id to its bare sequence number by keeping the first output line and stripping the server suffix, e.g. "98765.pbsserver" -> "98765".

Parameters:

• raw -- [in] Raw stdout captured from qsub.

int _knit_sched_pbs_submit()

Submit an already-written batch script with qsub and print the resulting job id. When the resolved "wait" flag is "true", qsub is run with "-W block=true" so it blocks until the job completes (its exit status becomes qsub's). The -o/-e redirection is carried by the script's directives, so the job directory is not needed here.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

• jobdir -- [in] Job directory (unused; redirection is set via directives).

int _knit_sched_pbs_submit_cmdline()

Build the qsub submission command into a caller-provided array, passed by name: "qsub [-W block=true] <script>". The "-W block=true" arguments are added when the resolved "wait" option is "true" (see _knit_sched_pbs_submit for its effect).

Parameters:

• argv_name -- [out] Name of the array to fill with the submission argv.

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

int _knit_sched_pbs_wait()

Block until PBS no longer runs the job. OpenPBS ships no qwait, so poll qstat every _KNIT_SCHED_POLL_INTERVAL seconds. -x also reports finished jobs from history when it is enabled. A job that is gone (unknown/purged, so no job_state line) is treated as finished; a job whose job_state is "E" (exiting, its Exit_status is already set) or "F" (finished) is terminal. "E" is treated as terminal because a job can linger in it after its script has stopped running, and waiting for "F" would then block far longer than the job runs. The job's knit terminal state is read from the DB by the caller afterwards.

Parameters:

• jobid -- [in] PBS job id (from the job's .job.id).

sched_slurm.sh

Functions

int _knit_sched_slurm_cancel()

Cancel a Slurm job with scancel. scancel sends SIGTERM (then SIGKILL after a grace period), which lets the batch shell's pre-termination handler record the job "killed" (see _knit_job_killed_trap). scancel exits 0 even for a job that has already finished, so no special-casing is needed here.

Parameters:

• jobid -- [in] Slurm job id (from the job's .job.id).

int _knit_sched_slurm_directives()

Emit the #SBATCH directive lines for a resolved job. Knit allocates whole nodes exclusively, so --exclusive is always set and --ntasks-per-node is pinned to the derived per-node core count (when known) so every core is a launchable slot. Job stdout/stderr are fixed to <jobdir>/.stdout and <jobdir>/.stderr. Optional fields (account, project/wckey, partition, gpus) are emitted only when set.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• jobdir -- [in] Job directory (for the --output/--error paths).

int _knit_sched_slurm_hostfile()

Print the host list for a Slurm job, one hostname per line. Slurm exposes the allocation as a compressed nodelist ($SLURM_JOB_NODELIST, e.g. "node[01-03]") rather than a hostfile, so expand it with scontrol show hostnames, which prints one hostname per allocated node. This is a per-node list (not per-task): Slurm has no native per-slot hostfile, so a raw request yields the same per-node entries. If the nodelist is unset or the expansion fails (e.g. not running inside a Slurm job), warn and fall back to this machine's hostname.

int _knit_sched_slurm_parse_jobid()

Extract the numeric job id from sbatch's output line "Submitted batch job <N>". Falls back to the last whitespace-separated token.

Parameters:

• raw -- [in] Raw stdout captured from sbatch.

int _knit_sched_slurm_submit()

Submit an already-written batch script with sbatch and print the resulting job id. When the resolved "wait" flag is "true", sbatch is run with --wait so it blocks until the job completes (its exit status becomes sbatch's). The --output/--error redirection is carried by the script's directives, so the job directory is not needed here.

Parameters:

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

• jobdir -- [in] Job directory (unused; redirection is set via directives).

int _knit_sched_slurm_submit_cmdline()

Build the sbatch submission command into a caller-provided array, passed by name: "sbatch [--wait] <script>". The --wait flag is added when the resolved "wait" option is "true" (see _knit_sched_slurm_submit for its effect).

Parameters:

• argv_name -- [out] Name of the array to fill with the submission argv.

• arr_name -- [in] Name of the resolved-options associative array.

• script -- [in] Path to the batch script to submit.

int _knit_sched_slurm_wait()

Block until Slurm no longer lists the job as active. Slurm has no reliable blocking "wait for completion" primitive after submission: scontrol wait_job returns as soon as the job is allocated (not when it finishes), and sbatch --wait only applies at submit time. So poll squeue for the job id every _KNIT_SCHED_POLL_INTERVAL seconds until it produces no rows, which is true once the job has completed, failed, or been cancelled (a running or completing CG job still lists). The job's knit terminal state is read from the DB by the caller afterwards.

Parameters:

• jobid -- [in] Slurm job id (from the job's .job.id).

set.sh

Functions

int _knit_set_add()

Add one or more elements to a set. Each element that is not already present is appended to the set's companion order array, so iteration reflects the order in which elements were first added. Duplicates are ignored (neither re-added nor re-ordered). The order array is created lazily, so this works on sets declared directly with declare -gA as well as those created via _knit_set_new().

Example:

_knit_set_add MY_SET "Shane" "Matthieu" "Rob"

Parameters:

• set_name -- [in] Name of the set in which to add the element.

• ...items -- [in] Elements to add to the set.

int _knit_set_array()

Return a set's elements, in insertion (declaration) order, as an indexed array written through a nameref output parameter. This is the fork-free alternative to while read … < <(_knit_set_iter …).

Example:

local -a members
_knit_set_array members MY_SET

Parameters:

• out -- [out] Name of the array variable to populate (nameref output).

• set_name -- [in] Name of the set to read.

int _knit_set_exists()

Checks if a set with the given name is defined, i.e. the variable is defined and it is an associative array.

Example:

_knit_set_exists MY_SET

Parameters:

• set_name -- [in] Name of the set.

int _knit_set_find()

Check if an element exists in a set.

Example:

_knit_set_find MY_SET "Phil"

Parameters:

• set_name -- [in] Name of the set in which to search.

• item -- [in] Item to find.

Returns:

0 if the element is found, 1 otherwise.

int _knit_set_iter()

Print each element of a set on its own line, in insertion (declaration) order.

Example:

_knit_set_iter MY_SET | while read -r key; do
    echo "Key: $key"
done

Parameters:

• set_name -- [in] Name of the set to iterate over.

int _knit_set_new()

Create a new empty set (associative array).

A set remembers the order in which elements were first added: alongside the backing associative array NAME, a companion indexed array NAME__order records insertion order so that _knit_set_iter() and _knit_set_array() yield elements in declaration order rather than the arbitrary hash order of the associative array's keys.

Example:

_knit_set_new MY_SET

Parameters:

• set_name -- [in] Name of the set to create.

int _knit_set_remove()

Remove one or more elements from a set, keeping the companion order array consistent.

Example:

_knit_set_remove MY_SET "Shane" "Matthieu"

Parameters:

• set_name -- [in] Name of the set from which to remove the elements.

• ...items -- [in] Elements to remove from the set.

setup.sh

Functions

int _knit_default_setup()

Body of the builtin "default" setup. It builds nothing: the setup exists only to carry the platform activation (inlined into .activate.sh by _knit_setup_default_after_cb) to jobs that declare no setup of their own.

int _knit_default_setup_path()

Print the directory where bootstrap instantiates the builtin "default" setup: <setup-root>/default, i.e. the reserved "default" instance under the experiment's setup root (see _knit_setup_root). It travels with the experiment (relative setup roots resolve against the experiment root) and survives the compute-side cd into a job directory. This is the path knit_with_setup "default" and the implicit-default job adoption resolve to when no --setup is given.

int _knit_experiment_root()

Store the experiment root — the directory that contains .knit — in the caller-named variable. Derived from _KNIT_PREFIX (already absolute) by stripping the trailing "/.knit" component, so it stays correct even after a compute-side cd into a job directory. Uses parameter expansion (no fork).

Parameters:

• __knit_ret -- [out] Name of the variable to hold the experiment root.

int _knit_has_user_setup()

Return 0 when at least one user setup instance exists directly under the given setup root, non-zero otherwise. A setup instance is a subdirectory of the root; the builtin "default" instance is not a user setup and is ignored, so a root that holds only "default" (or nothing) reports "no user setup". A missing root also reports "no user setup".

Parameters:

• root -- [in] Absolute setup root to scan (see _knit_setup_root).

int _knit_highlight_if_no_user_setup()

Highlight predicate (see knit_highlight_if) for the builtin "setup" command: return 0 ("highlight") when the experiment is bootstrapped but no setup other than the builtin "default" has been instantiated yet, non-zero otherwise. This bolds "setup" in the root "--help" once bootstrap is done — pointing at the natural next step — and leaves it plain as soon as the user has built a setup. Before bootstrap it never highlights (the setup root does not exist yet, and the command is filtered from "--help" anyway).

Parameters:

• cmd -- [in] The demangled command name (unused; the predicate is state-only).

int _knit_job_root()

Store the resolved job root — the directory under which jobs live — in the caller-named variable. Reads the verbatim job_path from the metadata table (falling back to "jobs" when unset, for robustness) and resolves it against the experiment root via _knit_resolve_experiment_path.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved job root.

int _knit_resolve_experiment_path()

Resolve a stored path value into an absolute directory and store it in the caller-named variable. An absolute value (starting with "/") is returned unchanged; a relative value is resolved against the experiment root, so an experiment with relative roots stays portable across machines and clones.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved path.

• stored -- [in] The stored path value (as typed at bootstrap).

int _knit_setup()

Entry point for the setup CLI command. Materializes a setup instance named --name under the experiment's setup root (<setup-root>/<name>), exports KNIT_SETUP_PREFIX to that directory, invokes the named setup subcommand inside it, and saves the resulting environment to $KNIT_SETUP_PREFIX/.activate.sh. On success the setup type is also recorded in $KNIT_SETUP_PREFIX/.setup.type so knit submit can validate a job's knit_with_setup requirement, and the setup body's row id in $KNIT_SETUP_PREFIX/.setup.id so a consumer can record a "used_by" edge to it. Removes the directory and fatals on failure.

The instance name must be a single path component (validated) and is idempotent by name: re-running with the same name rebuilds the same directory. The name "default" is reserved for the builtin default setup type.

Usage:

./exp.sh setup --name <name> -- <setup-type> [args...]

int _knit_setup_after_cb()

After-callback installed on every setup subcommand by knit_register_setup. Inlines the platform activation (${KNIT_PREFIX}/platform.sh) at the top of , then writes the activation lines the setup body declared through the knit_setup_env* / knit_setup_activate_line functions, in call order. A job depending on this setup sources that file to reproduce the build environment. Inlining the platform (rather than sourcing) keeps .activate.sh self-contained and re-initializes the module system for consumers, above the declared lines.

The declared lines are composable: an append/prepend extends the job's own variable rather than replacing it, so nothing snapshots the login-node state. A setup that declares nothing produces a header-only .activate.sh. The Spack and launcher after-callbacks (registered after this one) append their blocks last, so activation order is header -> declared lines -> spack -> launcher.

int _knit_setup_before_cb()

Before-callback installed on every setup subcommand by knit_register_setup. Verifies that KNIT_SETUP_PREFIX is set, ensuring the setup was invoked through knit setup rather than called directly, then sources the platform environment so the setup body builds against the platform's modules and env. Also clears the declared-activation-line array so each setup invocation starts with an empty set of lines.

int _knit_setup_check_type()

Check that a setup directory was built by the required setup type. knit setup records the type in a .setup.type marker file inside the setup directory. Fatals if the marker is missing or its recorded type differs from the required one; returns normally on a match. Shared by knit submit (jobs) and by the generic setup-dependency before-callback used for every other command that declares knit_with_setup.

Parameters:

• setup_path -- [in] Path to the setup directory to check.

• required -- [in] Setup type the directory must have been built by.

int _knit_setup_default_after_cb()

After-callback for the builtin "default" setup. Unlike the generic setup after-callback it does NOT dump the environment: the default setup builds nothing, so re-exporting the bootstrap shell's environment would freeze unrelated login-node state into every job. It writes only the platform activation header, so .activate.sh carries the platform modules/environment (and is effectively empty when there is no profile).

int _knit_setup_dep_after_cb()

After-callback installed by knit_with_setup on a plain command or an app, alongside _knit_setup_dep_before_cb. It records a "used_by" edge from the setup the command depends on to the command itself. It runs as an after-callback (not the before-callback that sources the environment) because the consumer's frame is on the executing stacks only from push time onward, so the consumer's resolved row id — the edge target — is available here but not in the before-callback.

It resolves the same setup directory the before-callback used (via _knit_setup_dep_resolve_path), so the two always agree.

Parameters:

• required -- [in] Setup type the dependency must have been built by. Remaining arguments are the command's own runtime arguments, scanned for --setup.

int _knit_setup_dep_before_cb()

Before-callback installed by knit_with_setup on a plain command or an app. It resolves the setup directory the command depends on, checks its type, and sources its .activate.sh so the command body runs in the setup's environment. Jobs do not use this callback: their setup is resolved at submit time (_knit_submit) and re-sourced on the compute node (_knit_job_before_cb). Setups and wrappers cannot declare knit_with_setup at all (knit_with_setup rejects them).

The setup directory is resolved by _knit_setup_dep_resolve_path (the --setup option, else the ambient KNIT_SETUP_PREFIX, else the builtin default path when the required type is "default").

Parameters:

• required -- [in] Setup type the dependency must have been built by. Remaining arguments are the command's own runtime arguments, scanned for --setup.

int _knit_setup_dep_resolve_path()

Resolve the setup directory a knit_with_setup command depends on, by precedence: the command's --setup option (a setup instance name, resolved to <setup-root>/<name> via _knit_setup_name_to_path), else an already-set KNIT_SETUP_PREFIX (an absolute path, e.g. an app running inside a job inherits the job's setup), else — only when the required type is the builtin "default" — the auto-instantiated default setup path. Prints the resolved path (empty when nothing resolves). Shared by the before- and after-callbacks so they always agree on which directory is referenced.

Parameters:

• required -- [in] Setup type the dependency must have been built by. Remaining arguments are the command's own runtime arguments, scanned for --setup.

int _knit_setup_name_to_path()

Resolve a setup instance name into its absolute directory under the experiment's setup root: <setup-root>/<name> (see _knit_setup_root). The name is validated as a single path component first (fatal otherwise). This is how a --setup value is resolved for jobs, plain commands, and apps: users refer to a setup by the name they gave knit setup --name, not by its on-disk path.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved setup directory.

• name -- [in] The setup instance name (as passed to --setup).

int _knit_setup_provides_launcher_after_cb()

After-callback installed by knit_provides_launcher, registered after the generic _knit_setup_after_cb (and any Spack re-activation block) so it appends to an already-written .activate.sh. It runs in the setup's own process, right after the body built and PATH-prepended its MPI, so the launcher is present on PATH by construction. It clears the detection cache, detects the launcher once against the active PATH, and freezes the concrete result by appending export KNIT_PROVIDED_LAUNCHER=<impl> to .activate.sh — the contract read by the launcher precedence (_knit_launch_backend). The value is also recorded as provenance via the mpi_launcher output.

If detection finds no launcher ("<unknown>") the setup declared it may provide one but built none reachable on PATH. Whether that is fatal depends on the machine: knit_provides_launcher means only "I may supply a launcher where the

machine has none", and the machine's own launcher (the profile's

launcher) wins over the setup's contract in _knit_launch_backend. So:

• When the machine has a concrete launcher (e.g. PALS on a Cray, whose mpiexec comes from a launcher module, not from the MPI the setup builds against), the setup's contract is subordinate and unused; a failed detection is expected and harmless. Skip the contract, record <unknown> provenance, and continue — a portable setup that unconditionally declares knit_provides_launcher is correct here (the profile wins).

• When the machine has no launcher (a laptop; launcher is <unknown>, "none", or unset), the setup's contract is the only possible source, so a failed detection is fatal — an early, clear failure rather than a silent degrade to the "none" backend at run time.

Trailing arguments are the setup's runtime arguments and are ignored.

int _knit_setup_record_used_by_edge()

Record a "used_by" provenance edge from the setup built at <setup_path> to a consuming invocation. The edge's source is the setup (its row id read from the setup directory's .setup.id marker, its name "setup:<type>" from .setup.type); its target is the consumer. A "used_by" edge has no duration, so both timestamps are NULL. Shared by knit submit (jobs) and by the generic setup-dependency after-callback (plain commands and apps).

It reads the setup directory's .setup.id / .setup.type markers and delegates the gated write to _knit_record_used_by_edge (which records nothing when recording is disabled, on a suppressed rank, before bootstrap, or when the target does not participate in the graph). A setup directory with no .setup.id (e.g. built before provenance shipped) yields no edge.

Parameters:

• setup_path -- [in] Path to the setup directory the consumer references.

• target_cmd -- [in] Mangled command name of the consumer (the edge target).

• target_id -- [in] Resolved row id of the consumer (the edge target).

int _knit_setup_require_body()

Guard shared by the declarative activation functions (knit_setup_env_* / knit_setup_activate_line): fatal unless the caller runs from inside a setup body. A setup body is the case where the currently executing command has type "setup"; this excludes a job body (which also sets KNIT_SETUP_PREFIX, because it sources the setup's environment) and any use outside a registered command.

Parameters:

• fn -- [in] Name of the calling function, used in the error message.

int _knit_setup_root()

Store the resolved setup root — the directory under which setup instances live — in the caller-named variable. Reads the verbatim setup_path from the metadata table (falling back to "setups" when unset, for robustness) and resolves it against the experiment root via _knit_resolve_experiment_path.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved setup root.

int _knit_setup_source_platform()

Source the materialized platform fragment (${_KNIT_PREFIX}/platform.sh) into the current shell so a setup builds with the platform's modules and environment (e.g. mpicc/cmake from modules) already active. A guarded no-op when the file is absent (no profile, or the profile declared neither modules nor environment). Shared by the generic setup before-callback and the Spack-env before-callback.

int _knit_setup_spack_env_after_cb()

After-callback installed by knit_with_spack_env, registered after the generic _knit_setup_after_cb so it runs last and its appended block is authoritative when a job sources .activate.sh. It (1) appends an explicit re-activation block to .activate.sh so a job re-activates the exact environment, and (2) records the concrete manifest (spack.yaml) and lockfile (spack.lock) as provenance outputs, gzip-compressed and base64-encoded.

The provenance is emitted with knit_output: after-callbacks run while the command is still on the executing-command stack (it is popped only afterwards), so knit_output resolves and type-checks the outputs normally.

Trailing arguments are the setup's runtime arguments and are ignored.

int _knit_setup_spack_env_before_cb()

Before-callback installed by knit_with_spack_env. Runs as the setup's first step (before-cbs execute in the setup's own shell, so the activated environment persists into the setup body and into the after-callbacks). It provisions Spack on demand if it is missing (see _knit_spack_ensure_provisioned), materializes the environment manifest to ${KNIT_SETUP_PREFIX}/spack.yaml, builds the Spack environment at ${KNIT_SETUP_PREFIX}/spack-env, and activates it in the current shell so the setup body builds against the installed packages.

The manifest source is captured at registration time and carried in the callback arguments (mode + source); trailing arguments are the setup's own runtime arguments and are ignored.

Parameters:

• mode -- [in] "file" (source is an absolute path to copy) or "stdin" (source is the literal manifest content captured from a here-doc).

• source -- [in] Manifest path (mode=file) or manifest content (mode=stdin).

int _knit_setup_validate_var()

Validate that a variable name given to a declarative activation function is a valid shell identifier (^[A-Za-z_][A-Za-z0-9_]*$). Fatal otherwise; returns normally on a match.

Parameters:

• var -- [in] The variable name to validate.

int _knit_setup_write_activate_header()

Print the header of a setup's .activate.sh to stdout: the shebang, a generated marker, and — when the materialized platform fragment (${_KNIT_PREFIX}/platform.sh) exists — its contents inlined. Inlining (rather than sourcing) keeps .activate.sh self-contained and re-initializes the module system for consumers. When there is no profile the platform fragment is absent and only the shebang/marker are emitted. Shared by the generic setup after-callback (which appends the setup's declared activation lines) and the builtin "default" setup's after-callback (which emits nothing more).

int _knit_stdin_is_terminal()

Return success if standard input is an interactive terminal. Factored out of knit_with_spack_env so tests can stub it: a real terminal is unavailable under bats, so the no-manifest guard cannot be exercised without this seam.

int _knit_validate_instance_name()

Validate a setup or job instance name. A name must be a single path component: it matches ^[A-Za-z0-9._-]+$ (letters, digits, dot, underscore, hyphen; no slashes). Fatals with guidance on anything else; returns normally on a match.

Parameters:

• name -- [in] The instance name to validate.

Variables

AssociativeArray _KNIT_SETUPS

Associative array mapping registered setup names to 1. Used to validate that a setup name passed to knit setup is known. Declared global (-g) so the builtin "default" entry, populated at load time, survives even when knit.sh is sourced from within a function (as the bats tests do).

Array _KNIT_SETUP_ACTIVATE_LINES

Per-invocation array of activation lines a setup body declared through the knit_setup_env_* / knit_setup_activate_line functions. Reset to empty at the start of each setup's execution (in _knit_setup_before_cb) and read by the generic setup after-callback, which writes the lines into $KNIT_SETUP_PREFIX/.activate.sh in call order. Declared global (-g) so the body functions and the after-callback share the same array.

shorthand.sh

Functions

int _knit_shorthand_find_function()

Discover the name a name-extracting registration shorthand (@command, @job, @app, @setup, @wrapper) decorates. The shorthand captures its own call site and passes it here; the name is read from the source rather than taken as an argument.

The source file is read and the lines after "line" are scanned. Blank lines, comments, and intervening decorator lines (@with_*, @usable_if, ...) are skipped. The scan stops at the first of:

• an "@empty" marker line -> the discovered body name is "knit_empty";

• a function definition -> its name is returned. All common styles are recognised: "name() {", "name () ...", "function name ...", one-liners, and names containing "@" (bash allows them).

The discovery is textual, so the function need not be defined yet when the shorthand runs.

Fatal when the source is not a readable file (piped/eval'd script), and when neither a function definition nor an "@empty" marker is found before the next "@done"/"knit_done" or the end of the file.

Parameters:

• The -- [out] name of the variable that receives the discovered name.

• file -- [in] The source file to read (BASH_SOURCE of the call site).

• line -- [in] The 1-based line number of the shorthand call (BASH_LINENO).

int _knit_shorthand_generate()

Define the "@" shorthand functions, honouring KNIT_WITHOUT_SHORTHAND. Runs once when knit.sh is sourced. For each non-opted-out pass-through token it defines "@<token>() { <target> "$"; }"; for each non-opted-out extracting

token it defines a wrapper that discovers the decorated function's name and

injects it at argument position 2. Any KNIT_WITHOUT_SHORTHAND token that is not

a real shorthand produces a warning and is otherwise ignored; the special

value "all" suppresses every shorthand.

Variables

AssociativeArray _KNIT_SHORTHAND_EXTRACTOR

Curated map from a name-extracting shorthand token to the knit_register* function it forwards to. These discover the decorated function's name from the source and inject it at argument position 2. The wrappers are generated in a later step; the map is declared here so opt-out validation already recognises these tokens as real shorthands.

AssociativeArray _KNIT_SHORTHAND_PASSTHROUGH

The "@" shorthand layer. This file defines a terse, decorator-style twin for every declaration/decoration function in the public API: "knit_x" gains an "@x" twin (with the register family remapped: knit_register -> @command, knit_register_<x> -> <x>). The shorthand is additive — the knit_* functions are unchanged and remain the canonical, stable API — and it is enabled by default. A user opts out by setting KNIT_WITHOUT_SHORTHAND before sourcing knit (a comma-separated list of tokens, or "all").

The curated set of shorthands lives in the two maps below. It cannot be derived mechanically from the knit_* prefix, because that prefix also covers runtime helpers (knit_get_parameter, the loggers, ...) which get no shorthand.

Pass-through shorthands forward their arguments verbatim to the target (@x() { knit_x "$@"; }). The extracting shorthands (the register family) are wired up in a later step: they discover the decorated function's name from the source rather than taking it as an argument.

Curated map from a pass-through shorthand token (the part after "@") to the knit_* function it forwards to verbatim. These take no function-name argument and are not name-extracting: the generated wrapper simply forwards "$@".

skills.sh

Functions

int _knit_skills_download()

Download the knit repository at a ref as a tarball and extract it into the destination directory, so <dest>/agent/ holds the skills and commands. Uses curl and tar (as the sqlite/jq/spack provisioning does) so knit needs no git, and --strip-components=1 drops the archive's single "knit-<ref>/" top-level directory without having to resolve the ref to a commit first. GitHub serves any ref at archive/<ref>.tar.gz.

Parameters:

• dest -- [in] Destination directory (already created by the caller).

• ref -- [in] Git ref (branch, tag, or commit SHA) to download.

int _knit_skills_install()

Implementation of 'knit skills install'. Downloads the latest agent/ tree from the knit repository and installs the skills and commands into ".agents/", the canonical cross-harness location. The copy merges into any existing skills/commands directory (so other, non-knit skills there are preserved) and overwrites knit's own, which makes a re-install idempotent.

With --claude, Claude Code is pointed at that same install by symlinking knit's skills and commands into ".claude" one item at a time (see _knit_skills_link_claude), so there is one real copy on disk, not a per-harness fork, and a project's own ".claude" items are left in place.

It also drops the agent/AGENTS.md pointer at the project root so an agent orients itself even before a skill is loaded. An existing project AGENTS.md is never overwritten (it may hold the user's own notes); the pointer is written only when no AGENTS.md is present.

int _knit_skills_link_claude()

Point Claude Code at the canonical ".agents" install by symlinking knit's items into ".claude/skills" and ".claude/commands", one entry at a time. The two containers are kept as real directories (created if missing, and an existing one — even a symlink — is left as is), so a project's own skills/commands and its ".claude/settings.json" live alongside the links rather than being replaced.

Each immediate entry of ".agents/skills" (a skill directory) and ".agents/commands" (a command file) is symlinked into the matching container. The target is absolute (${PWD}/.agents/...) so the link is valid wherever the container physically lives — in particular when the container is itself a symlink, where a relative target would resolve from the wrong place. An entry name already present is skipped with a warning rather than overwritten, so a user's own item is never clobbered; an entry that is already the exact link this would create is skipped silently, which makes a re-install idempotent.

Variables

String _KNIT_SKILLS_DEFAULT_REF

Git ref the install resolves against when no --ref is given. This is the default branch, not the running knit version tag, because the agent/ tree on the default branch carries the most up-to-date skills (mirrors the profile store convention).

String _KNIT_SKILLS_REPO

GitHub repository (org/repo) that serves the agent skills and commands tree under agent/. The install downloads this repository's archive.

spack.sh

Functions

int _knit_bootstrap_need_spack()

Decide whether Spack must be provisioned during bootstrap. Spack is needed when the user gave a non-empty --spack or --spack-packages ref, or when a registered setup declared a Spack environment (_KNIT_SPACK_REQUIRED).

Parameters:

• spack_ref -- [in] Value of the --spack option (may be empty).

• packages_ref -- [in] Value of the --spack-packages option (may be empty).

Returns:

0 if Spack is needed, 1 otherwise.

int _knit_bootstrap_spack()

Provision a knit-private Spack: resolve refs (latest release when empty) to exact commits, download and extract Spack and spack-packages with curl+tar, write a local-path repos.yaml, and record provenance metadata (requested refs and resolved commits). No git dependency.

Parameters:

• spack_ref -- [in] Spack ref (tag/branch/commit); empty uses the latest release.

• packages_ref -- [in] spack-packages ref; empty uses the latest release.

int _knit_bootstrap_spack_env_built()

Return 0 when at least one setup instance has a built Spack environment, non-zero otherwise. A built environment is a "spack-env/spack.lock" lockfile under a setup instance directory (see _knit_setup_spack_env_before_cb). Used by update-mode Spack handling to freeze the Spack version once any environment has been concretized, for reproducibility. A missing setup root reports "none

built".

int _knit_bootstrap_update_spack()

Update-mode handler for the Spack options (--spack, --spack-packages). Spack anchors reproducibility, so its version is frozen once an environment is built:

• Spack not provisioned yet: provision it now when a ref was typed or a Spack-requiring setup is registered (the first-bootstrap condition). This adds Spack that the first bootstrap did not.

• Spack already provisioned and a typed ref differs from the stored ref:

  • no Spack environment is built anywhere -> re-provision at the new ref(s) and update the stored ref/commit metadata;

  • a Spack environment is built -> fatal, for reproducibility.

• A typed ref equal to the stored ref (or no ref typed) is a no-op.

An untyped ref keeps its stored value across a re-provision, so changing only --spack does not silently move spack-packages to the latest release.

Parameters:

• spack_ref -- [in] Value typed for --spack (may be empty).

• packages_ref -- [in] Value typed for --spack-packages (may be empty).

• ... -- [in] Raw argument tokens of this invocation (see _KNIT_INVOCATION_RAW_ARGS), used to tell a typed option from a defaulted one.

Returns:

0 when Spack was provisioned or re-provisioned, 1 when nothing changed.

int _knit_spack()

int _knit_spack_download()

Download a Spack repository at a specific commit as a tarball and extract it into the destination directory. Uses curl and tar (as the sqlite/jq provisioning does) so knit has no git dependency. GitHub serves any commit at archive/<sha>.tar.gz, whose single top-level "<repo>-<sha>/" directory is stripped so files land directly in the destination.

Parameters:

• repo -- [in] Repository name under the spack org ("spack" or "spack-packages").

• dest -- [in] Destination directory.

• sha -- [in] Commit SHA to download.

int _knit_spack_ensure_provisioned()

Ensure the knit-private Spack is present, provisioning it on demand when it is absent. Bootstrap provisions Spack only when a Spack-backed setup was declared at bootstrap time (or --spack was given). A setup that gains a knit_with_spack* directive after bootstrap would otherwise find no Spack and, since bootstrap cannot be re-run, force the user to delete and recreate .knit by hand. Instead this downloads Spack at first use — the latest release, as an empty --spack would — announced with a knit_info line so the one-time delay is not a mystery.

A no-op when Spack is already provisioned. Fatal when the experiment is not bootstrapped: there is no .knit to provision into, and _knit_bootstrap_spack's provenance writes need the metadata table.

int _knit_spack_env_install()

Create and install a Spack environment from a manifest. The environment is created as a directory ("anonymous") environment at <env-dir> from the given spack.yaml, then its specs are installed. Both steps run through _knit_spack_exec, so the knit-private Spack is used and setup-env.sh is sourced at most once per process. The (long-running) install is framed.

When the profile declares Spack config, "${_KNIT_PREFIX}/spack-config.json" exists (the profile's spack object wrapped under a top-level spack key); it is merged into the environment with "spack config add -f" before concretization, so specs like "mpi"/"hdf5" resolve to the platform's vendor installs and provider preferences apply. The step is skipped when the file is absent.

The install runs directly (its stdout/stderr inherit the caller's terminal) so Spack's own TTY-aware progress output is shown, rather than piped through a frame (which both hid it behind trace level and would defeat Spack's TTY detection). Every step is checked: on failure the function logs a knit_error and returns non-zero (it does NOT knit_fatal) so the setup dispatcher can remove the half-built setup directory and avoid recording it.

Parameters:

• env_dir -- [in] Directory in which to create the Spack environment.

• yaml -- [in] Path to the spack.yaml manifest describing the environment.

Returns:

0 if the environment was created and installed, non-zero otherwise.

int _knit_spack_exec()

Run the knit-private Spack, forwarding all arguments verbatim (this is the body of the "knit spack" wrapper). Fatal-with-hint if Spack has not been provisioned. Spack's setup-env.sh is sourced at most once per process (guarded by _KNIT_SPACK_ENV_SOURCED) so that a script calling "knit spack" repeatedly pays the sourcing cost only on the first call. Note: we deliberately do not 'exec spack' — that would replace the caller's shell and prevent any later "knit spack" from running; instead we call the (sourced) spack function and return its exit status.

Parameters:

• ... -- [in] Arguments forwarded verbatim to spack (including --help).

Returns:

The exit status of spack.

int _knit_spack_framed_run()

Run a command with its combined stdout/stderr written to _KNIT_TRACE_FILE and, when KNIT_LOG_LEVEL is trace, also displayed live in a 10-line frame. Returns the exit status of the command.

Parameters:

• title -- [in] Title shown on the frame's top border.

• ... -- [in] Command and arguments to execute.

int _knit_spack_github_api()

Fetch a GitHub REST API URL and print the response body. When GITHUB_TOKEN or GH_TOKEN is set, the request is authenticated: the anonymous api.github.com limit is 60 requests per hour per address, shared by every process behind that address, so continuous-integration runners exhaust it and knit sees an error body instead of JSON. An authenticated request uses the far higher per-token limit. curl also retries a few times so a transient network error does not abort a bootstrap, and "-f" makes an HTTP error status a clean failure (empty output) instead of a non-JSON body that would confuse the caller's parser.

Parameters:

• url -- [in] The api.github.com URL to fetch.

Returns:

Prints the response body; non-zero on a hard failure or HTTP error.

int _knit_spack_install()

Install the specified specs using spack.

Parameters:

• ... -- [in] Specs to install.

int _knit_spack_latest_release()

Resolve the latest release tag of a Spack repository via the GitHub API. Used when the user did not pin a ref: knit provisions the newest published release.

Parameters:

• repo -- [in] Repository name under the spack org ("spack" or "spack-packages").

Returns:

Prints the newest release tag; fatal if none can be resolved.

int _knit_spack_resolve_commit()

Resolve a ref (tag, branch, or commit SHA) to its exact commit SHA via the GitHub API. Resolving upstream gives a single download path (the archive URL takes a SHA) and the exact commit for provenance, without needing git.

Parameters:

• repo -- [in] Repository name under the spack org ("spack" or "spack-packages").

• ref -- [in] Tag, branch, or commit SHA to resolve.

Returns:

Prints the commit SHA; fatal if it cannot be resolved.

int _knit_spack_write_repos_yaml()

Write <spack-root>/etc/spack/repos.yaml pointing the builtin package repo at the already-extracted spack-packages tree by local filesystem path. The local-path form needs no git: a git-backed "destination" would make Spack treat the tree as a clone (which the tarball extraction is not) and reach for git at runtime. Reproducibility comes from the pinned commit downloaded into that tree, recorded in provenance metadata.

Variables

String _KNIT_SPACK_ENV_SOURCED

Guard so Spack's setup-env.sh is sourced at most once per process. The wrapper runs in the caller's own shell (no subshell, no exec), so the first "knit

spack" invocation sources setup-env.sh and sets this flag; subsequent invocations in the same script reuse the already-modified PATH and the spack shell function instead of paying the (slow) sourcing cost again. Empty means "not yet sourced".

String _KNIT_SPACK_PACKAGES_ROOT

Root directory for the pre-cloned spack-packages repository (Spack >= 1.0 keeps package recipes in a separate repo, referenced by repos.yaml).

String _KNIT_SPACK_REQUIRED

Set to a non-empty value at registration time (by knit_with_spack_env) when a setup declares a Spack environment, so bootstrap auto-provisions Spack even without an explicit --spack ref. Empty means "not required".

String _KNIT_SPACK_ROOT

Root directory for the Spack installation.

sqlite.sh

Variables

String _KNIT_DATABASE

Path to the Knit database file.

String _KNIT_SQLITE_EXE

Path to the SQLite executable.

String _KNIT_SQLITE_SOURCE_NAME

Name of the SQLite source archive.

String _KNIT_SQLITE_SOURCE_URL

URL to download the SQLite source archive.

str.sh

Functions

int _knit_str_hyphens_to_underscores()

Convert hyphens to underscores, storing the result in the caller-named variable.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the result.

• input -- [in] String to convert.

int _knit_str_render_cmd()

Render an argument vector, passed by array name, as a single shell-safe string: each element is q-quoted and the elements are joined by single spaces. An empty array renders to the empty string. Used to capture the resolved launcher and scheduler commands for the "native_cmd" columns of the runs and jobs tables and for the trace log emitted before each command is issued.

Parameters:

• argv_name -- [in] Name of the array holding the command and its arguments.

int _knit_str_underscores_to_hyphens()

Convert underscores to hyphens, storing the result in the caller-named variable.

Parameters:

• __knit_ret -- [out] Name of the variable to hold the result.

• input -- [in] String to convert.

types.sh

Functions

int _knit_type_check_date()

Validate that a string is a date in YYYY-MM-DD format with valid ranges.

Parameters:

• value -- [in] String to validate.

Returns:

0 if valid, 1 otherwise.

int _knit_type_check_time()

Validate that a string is a time in hh:mm:ss format with valid ranges.

Parameters:

• value -- [in] String to validate.

Returns:

0 if valid, 1 otherwise.

int _knit_type_is_checksummable()

Return 0 if a type name (or alias) refers to a target whose content Knit checksums — the "file" and "directory" types (and the "dir" alias). Every other type, including "path" and "filename", is not checksummable. This is the single predicate the declaration and recording paths consult to decide whether a parameter or output gains a companion checksum column.

Parameters:

• type_name -- [in] Type name or alias to test.

Returns:

0 if checksummable, 1 otherwise.

int _knit_type_resolve_alias()

Resolve a type name or alias to its canonical type name. If the name is already a canonical built-in type or an enum, it is returned as-is. If it is an alias, the corresponding canonical name is printed.

Example:

local t; _knit_type_resolve_alias t "int"      # t == "integer"
local t; _knit_type_resolve_alias t "integer"  # t == "integer"
local t; _knit_type_resolve_alias t "color"    # t == "color" (if enum defined)

Parameters:

• __knit_ret -- [out] Name of the variable to hold the resolved type name.

• type_name -- [in] Type name or alias to resolve.

Returns:

0 if resolved successfully, 1 if the name is unknown.

int _knit_type_to_sqlite()

Map a Knit type name (or alias) to its corresponding SQLite type affinity. Returns INTEGER for integer, REAL for real, and TEXT for all other types (including boolean, string, path, file, directory, filename, date, time, datetime, uuid, and user-defined enums).

Example:

local t; _knit_type_to_sqlite t "integer"  # t == INTEGER
local t; _knit_type_to_sqlite t "real"     # t == REAL
local t; _knit_type_to_sqlite t "uuid"     # t == TEXT
local t; _knit_type_to_sqlite t "int"      # t == INTEGER (alias resolved)

Parameters:

• __knit_ret -- [out] Name of the variable to hold the SQLite type affinity.

• type_name -- [in] Knit type name or alias.

Returns:

0 on success, 1 if the type is unknown.

Variables

AssociativeArray _KNIT_BUILTIN_ENUMS

Set of enum type names that have been marked as framework builtins (via _knit_is_builtin). Used to distinguish knit's own enums from user-defined ones.

AssociativeArray _KNIT_BUILTIN_TYPES

Set of built-in canonical type names.

AssociativeArray _KNIT_ENUMS

Set of user-defined enum type names.

String _KNIT_LAST_ENUM

Name of the most recently defined enum (set by knit_enum). Consulted by _knit_is_builtin when called outside a command registration, so a builtin enum can be marked immediately after its definition.

AssociativeArray _KNIT_TYPE_ALIASES

Associative array mapping type alias names to their canonical type names.