Python Runner API

Python Runner API#

The programmatic entry point for executing an app’s lifecycle. run_app runs the full dispatch → inputs → process → collect → register sequence for a workunit; Runner exposes the individual steps if you need finer control; ChunksFile describes the chunks discovered under a work directory.

When to use this vs. the CLI#

run_app is what bfabric-app-runner run workunit calls under the hood. Most users should drive the app runner through the CLI — reach for this API only when embedding execution in another Python program or orchestrator.

run_app#

bfabric_app_runner.app_runner.runner.run_app(app_spec: AppVersion, workunit_ref: int | Path, work_dir: Path, client: Bfabric, force_storage: Path | None, ssh_user: str | None = None, read_only: bool = False, dispatch_active: bool = True) → None#

Executes all steps of the provided app: dispatch, then inputs/process/collect for every chunk.

Unless read_only is set, the workunit status is set to processing before the run and to available once all chunks have been processed.

Parameters:

  • app_spec – Resolved app version whose commands and settings drive execution.

  • workunit_ref – Workunit to run, either a B-Fabric workunit ID or a path to a workunit definition YAML.

  • work_dir – Directory in which inputs, chunks, and outputs are staged.

  • client – B-Fabric client used to read the workunit and register outputs.

  • force_storage – Overrides the storage used for output registration; None uses the default storage.

  • ssh_user – SSH user for staging inputs and copying outputs; None uses the current user.

  • read_only – When True, skips all B-Fabric mutations (workunit status updates and output registration).

  • dispatch_active – When True, runs the dispatch step; set False to reuse an existing chunk layout.

Runner#

Runs a single app version’s steps against a work/chunk directory. Each method wraps one lifecycle stage; run_collect is a no-op when the app version defines no collect command.

class bfabric_app_runner.app_runner.runner.Runner(spec: AppVersion, client: Bfabric, ssh_user: str | None = None)#

Executes the individual lifecycle steps (dispatch, inputs, process, collect) of a single app version.

run_collect(workunit_ref: int | Path, chunk_dir: Path) → None#

Runs the app’s collect command for a chunk, or does nothing if the app has no collect step.

run_dispatch(workunit_ref: int | Path, work_dir: Path) → None#

Runs the app’s dispatch command, which splits the workunit into chunk directories under work_dir.

run_inputs(chunk_dir: Path) → None#

Stages the input files declared in the chunk’s inputs.yml into the chunk directory.

run_process(chunk_dir: Path) → None#

Runs the app’s process command on a single prepared chunk directory.

ChunksFile#

Represents the chunks.yml in a work directory. read loads it (auto-discovering and writing it when absent); infer_from_directory scans for subdirectories containing an inputs.yml.

class bfabric_app_runner.app_runner.runner.ChunksFile(*, chunks: list[Path])#

Lists the chunk subdirectories that make up a workunit’s work directory (chunks.yml).

classmethod infer_from_directory(work_dir: Path) → ChunksFile#

Infer chunks by scanning for subdirectories containing inputs.yml.

Parameters:

work_dir – The work directory to scan

Returns:

ChunksFile with discovered chunks

Raises:

ValueError – If no chunks are found

classmethod read(work_dir: Path) → ChunksFile#

Reads the chunks.yml file from the specified work directory.

If chunks.yml is missing, automatically discovers chunks by scanning for subdirectories containing inputs.yml and writes the result to chunks.yml.

Parameters:

work_dir – The work directory containing chunks.yml or chunk subdirectories

Returns:

ChunksFile with chunk paths

See Also#