Input specification#
The inputs module provides a specification schema to define the inputs required by an app.
You can also use this functionality interactively while prototyping.
The file is usually called inputs.yml and lists the different inputs, with information and how to retrieve them and
the filename to save them as.
General structure#
Generally the structure is a yaml file containing a key inputs which is a list of dictionaries, each representing an
input file.
Each input has a type key which identifies the input type.
This will allow us to extend this logic to different sources in the future.
In general the only other input key that will be available for all types is filename, which is the name of the file to
save the input as.
Fields like id might not be relevant for all types in the future, and depending on the type more specific options
might exist.
An example file could look like this:
# file: inputs.yml
inputs:
- type: bfabric_dataset
id: 53706
filename: test.csv
- type: bfabric_resource
id: 2700958
filename: test.zip
Commands#
Validation#
The input file can be validated with the command:
bfabric-app-runner validate inputs-spec inputs.yml
Which on success will output a pretty-printed version of the inputs file. Validation will also be performed by all other commands, so this is not strictly necessary.
For instance, in the above case this would print:
InputsSpec(
│ inputs=[
│ │ DatasetSpec(type='bfabric_dataset', id=53706, filename='test.csv', separator=','),
│ │ ResourceSpec(type='bfabric_resource', id=2700958, filename='test.zip', check_checksum=True)
│ ]
)
Here you can also see all the extra parameters which were implicitly set.
Prepare files#
The prepare command downloads your files and requires two arguments.
The first is the input file, and the second is the directory to save the files to.
In general to download to the current directory simply use . as the second argument:
bfabric-app-runner inputs prepare inputs.yml .
If your files already exist and are up-to-date, it will not download them again.
List files#
You can list the files that are present or will be downloaded:
bfabric-app-runner inputs list inputs.yml .
If you also want to check whether the files are up-to-date, you can pass the --check flag:
bfabric-app-runner inputs list --check inputs.yml .
Reference#
- pydantic model bfabric_app_runner.specs.inputs_spec.InputsSpec#
Bases:
BaseModelShow JSON schema
{ "title": "InputsSpec", "type": "object", "properties": { "inputs": { "items": { "discriminator": { "mapping": { "bfabric_annotation": { "discriminator": { "mapping": { "resource_sample": "#/$defs/BfabricAnnotationResourceSampleSpec" }, "propertyName": "annotation" }, "oneOf": [ { "$ref": "#/$defs/BfabricAnnotationResourceSampleSpec" } ] }, "bfabric_dataset": "#/$defs/BfabricDatasetSpec", "bfabric_order_fasta": "#/$defs/BfabricOrderFastaSpec", "bfabric_resource": "#/$defs/BfabricResourceSpec", "file": "#/$defs/FileSpec", "file_scp": "#/$defs/FileScpSpec", "static_yaml": "#/$defs/StaticYamlSpec" }, "propertyName": "type" }, "oneOf": [ { "$ref": "#/$defs/BfabricResourceSpec" }, { "$ref": "#/$defs/FileSpec" }, { "$ref": "#/$defs/FileScpSpec" }, { "$ref": "#/$defs/BfabricDatasetSpec" }, { "$ref": "#/$defs/BfabricOrderFastaSpec" }, { "discriminator": { "mapping": { "resource_sample": "#/$defs/BfabricAnnotationResourceSampleSpec" }, "propertyName": "annotation" }, "oneOf": [ { "$ref": "#/$defs/BfabricAnnotationResourceSampleSpec" } ] }, { "$ref": "#/$defs/StaticYamlSpec" } ] }, "title": "Inputs", "type": "array" } }, "$defs": { "BfabricAnnotationResourceSampleSpec": { "properties": { "type": { "const": "bfabric_annotation", "default": "bfabric_annotation", "title": "Type", "type": "string" }, "filename": { "pattern": "^[^/][^:]*$", "title": "Filename", "type": "string" }, "annotation": { "const": "resource_sample", "default": "resource_sample", "title": "Annotation", "type": "string" }, "separator": { "title": "Separator", "type": "string" }, "resource_ids": { "items": { "type": "integer" }, "title": "Resource Ids", "type": "array" } }, "required": [ "filename", "separator", "resource_ids" ], "title": "BfabricAnnotationResourceSampleSpec", "type": "object" }, "BfabricDatasetSpec": { "additionalProperties": false, "properties": { "type": { "const": "bfabric_dataset", "default": "bfabric_dataset", "title": "Type", "type": "string" }, "id": { "title": "Id", "type": "integer" }, "filename": { "pattern": "^[^/][^:]*$", "title": "Filename", "type": "string" }, "separator": { "default": ",", "enum": [ ",", "\t" ], "title": "Separator", "type": "string" } }, "required": [ "id", "filename" ], "title": "BfabricDatasetSpec", "type": "object" }, "BfabricOrderFastaSpec": { "additionalProperties": false, "properties": { "type": { "const": "bfabric_order_fasta", "default": "bfabric_order_fasta", "title": "Type", "type": "string" }, "id": { "title": "Id", "type": "integer" }, "entity": { "enum": [ "workunit", "order" ], "title": "Entity", "type": "string" }, "filename": { "pattern": "^[^/][^:]*$", "title": "Filename", "type": "string" }, "required": { "default": false, "title": "Required", "type": "boolean" } }, "required": [ "id", "entity", "filename" ], "title": "BfabricOrderFastaSpec", "type": "object" }, "BfabricResourceSpec": { "additionalProperties": false, "properties": { "type": { "const": "bfabric_resource", "default": "bfabric_resource", "title": "Type", "type": "string" }, "id": { "title": "Id", "type": "integer" }, "filename": { "anyOf": [ { "pattern": "^[^/][^:]*$", "type": "string" }, { "type": "null" } ], "default": null, "title": "Filename" }, "check_checksum": { "default": true, "title": "Check Checksum", "type": "boolean" } }, "required": [ "id" ], "title": "BfabricResourceSpec", "type": "object" }, "FileScpSpec": { "additionalProperties": false, "properties": { "type": { "const": "file_scp", "default": "file_scp", "title": "Type", "type": "string" }, "host": { "title": "Host", "type": "string" }, "absolute_path": { "title": "Absolute Path", "type": "string" }, "filename": { "anyOf": [ { "pattern": "^[^/][^:]*$", "type": "string" }, { "type": "null" } ], "default": null, "title": "Filename" } }, "required": [ "host", "absolute_path" ], "title": "FileScpSpec", "type": "object" }, "FileSourceLocal": { "properties": { "local": { "pattern": "^/[^:]*$", "title": "Local", "type": "string" } }, "required": [ "local" ], "title": "FileSourceLocal", "type": "object" }, "FileSourceSsh": { "properties": { "ssh": { "$ref": "#/$defs/FileSourceSshValue" } }, "required": [ "ssh" ], "title": "FileSourceSsh", "type": "object" }, "FileSourceSshValue": { "properties": { "host": { "title": "Host", "type": "string" }, "path": { "pattern": "^/[^:]*$", "title": "Path", "type": "string" } }, "required": [ "host", "path" ], "title": "FileSourceSshValue", "type": "object" }, "FileSpec": { "properties": { "type": { "const": "file", "default": "file", "title": "Type", "type": "string" }, "source": { "anyOf": [ { "$ref": "#/$defs/FileSourceSsh" }, { "$ref": "#/$defs/FileSourceLocal" } ], "title": "Source" }, "filename": { "anyOf": [ { "pattern": "^[^/][^:]*$", "type": "string" }, { "type": "null" } ], "default": null, "title": "Filename" }, "link": { "default": false, "title": "Link", "type": "boolean" } }, "required": [ "source" ], "title": "FileSpec", "type": "object" }, "StaticYamlSpec": { "properties": { "type": { "const": "static_yaml", "default": "static_yaml", "title": "Type", "type": "string" }, "data": { "anyOf": [ { "type": "object" }, { "items": {}, "type": "array" } ], "title": "Data" }, "filename": { "title": "Filename", "type": "string" } }, "required": [ "data", "filename" ], "title": "StaticYamlSpec", "type": "object" } }, "additionalProperties": false, "required": [ "inputs" ] }
- field inputs: list[InputSpecType] [Required]#
- classmethod read_yaml(path: Path) InputsSpec#
- classmethod read_yaml_old(path: Path) list[InputSpecType]#
- classmethod write_yaml(specs: list[InputSpecType], path: Path) None#
- apply_filter(filter: str, client: Bfabric) InputsSpec#