Output specification

The outputs module provides a specification schema to define the outputs that were created by an app and should be registered. The file is usually called outputs.yml and lists the different output files, with information how to register them.

General structure

Generally the structure is a yaml file containing a key outputs which is a list of dictionaries, each representing an output file. Each output has a type key which identifies the output type. This will allow us to extend this logic to different sources in the future.

An example file could look like:

outputs:
- type: bfabric_copy_resource
  local_path: /tmp/work/hello.txt
  store_entry_path: WU123456_hello.txt
- type: bfabric_dataset
  local_path: /tmp/work/hello.csv
  separator: ","
  name: Hello Dataset
- type: bfabric_link
  name: Results Report
  url: https://example.com/report/123

Commands

Validation

The output file can be validated with the command:

bfabric-app-runner validate outputs-spec outputs.yml

Which on success will output a pretty-printed version of the outputs file. Validation will also be performed by all other commands, so this is not strictly necessary.

Register files

To perform the registration to B-Fabric the following can be used:

bfabric-app-runner outputs register outputs.yml 1234

Please note:

• The workunit reference (an integer workunit ID or a path to a workunit definition YAML file) must be specified, so the correct information can be retrieved.

• Several actions might require a particular user to be possible, e.g. the bfabric_copy_resource will require a user with permission to create the particular file over SSH.

Reference

Each entry’s type field selects one of the models below.

Copy a local file into B-Fabric storage as a resource:

pydantic model bfabric_app_runner.specs.outputs_spec.CopyResourceSpec

Copies a local file into B-Fabric storage and registers it as a resource of the workunit.

Fields:

• local_path (Path)

• protocol (Literal['scp', 'tus'])

• store_entry_path (Path)

• store_folder_path (Path | None)

• type (Literal['bfabric_copy_resource'])

• update_existing (UpdateExisting)

field local_path: Path [Required]

The local path to the file to be copied.

field protocol: Literal['scp', 'tus'] = 'scp'

Transfer protocol used to copy the file to storage. Only "scp" is wired for app-runner; "tus" is accepted by the model but its output path is not yet implemented (raises NotImplementedError at registration).

field store_entry_path: Path [Required]

The path to the storage entry in the storage folder.

field store_folder_path: Path | None = None

The storage folder will be determined by the default rule, but can be specified if needed.

field type: Literal['bfabric_copy_resource'] = 'bfabric_copy_resource'

field update_existing: UpdateExisting = UpdateExisting.IF_EXISTS

Behavior if a resource with the same name already exists on the workunit.

Register a local table as a B-Fabric dataset. The format field selects the reader — csv (the default, a delimited text file) or parquet; separator and has_header apply to csv only:

pydantic model bfabric_app_runner.specs.outputs_spec.SaveDatasetSpec

Uploads a local file as the workunit’s output dataset.

Fields:

• format (Literal['csv', 'parquet'])

• has_header (bool)

• invalid_characters (str)

• local_path (Path)

• name (str | None)

• separator (str | None)

• type (Literal['bfabric_dataset'])

• update_existing (UpdateExisting)

field format: Literal['csv', 'parquet'] = 'csv'

Format of the local file. csv reads a delimited text file (see separator/has_header); parquet reads a Parquet file (separator/has_header are ignored).

field has_header: bool = True

Whether the local file’s first row contains column names. Applies to format: csv only.

field invalid_characters: str = ''

Characters that must not appear in any cell; upload fails if any occur (empty string disables the check).

field local_path: Path [Required]

Path to the local file to upload as the dataset.

field name: str | None = None

Name for the dataset in B-Fabric; None uses the local file’s stem.

field separator: str | None = None

Field separator of the local file for format: csv (e.g. a comma or a tab character); None defaults to a comma. Ignored for other formats.

field type: Literal['bfabric_dataset'] = 'bfabric_dataset'

field update_existing: UpdateExisting = UpdateExisting.IF_EXISTS

Behavior if the workunit already has an output dataset with the same name.

Attach a link to the workunit (or another entity):

pydantic model bfabric_app_runner.specs.outputs_spec.SaveLinkSpec

Saves a link to the workunit, or, if desired to an arbitrary entity of type entity_type with id entity_id.

Fields:

• entity_id (int | None)

• entity_type (str)

• name (str)

• type (Literal['bfabric_link'])

• update_existing (UpdateExisting)

• url (str)

field entity_id: int | None = None

The ID of the entity that will be linked.

field entity_type: str = 'Workunit'

The type of the entity that will be linked.

field name: str [Required]

The name of the link.

field type: Literal['bfabric_link'] = 'bfabric_link'

field update_existing: UpdateExisting = UpdateExisting.IF_EXISTS

Behavior, if a link with the same name already exists.

field url: str [Required]

The URL of the link.

The update_existing field on each output uses the shared UpdateExisting enum, which controls what happens when an output with the same name already exists:

class bfabric_app_runner.specs.outputs_spec.UpdateExisting(*values)

Policy for what to do when an output with the same identity already exists in B-Fabric.

Shared by the resource, dataset, and link output specs.

IF_EXISTS = 'if_exists'

Update the entry in place if it exists, otherwise create a new one.

NO = 'no'

Never touch an existing entry; fail if one already exists.

REQUIRED = 'required'

Update an existing entry, failing if none exists to update.