Changelog

The format is based on Keep a Changelog.

Versioning currently follows X.Y.Z semantic versioning. Historically, before 1.14.0, a different versioning scheme was used.

Minor breaking changes are still possible in 1.X.Y but we try to announce them with DeprecationWarnings in the previous Y-1 release.

[Unreleased]

Added

• Improved token authentication

  • bfabric.experimental.webapp_integration_settings

  • Bfabric.connect_token, Bfabric.connect_token_async: this one respects the list of allowed bfabric instances

  • bfabric.rest.token_data.validate_token

  • Bfabric.from_token_data: creates a new Bfabric instance from token data.

• ResultContainer.to_polars now has a flatten parameter to flatten struct columns into individual columns.

Deprecated

• Bfabric.from_config

• Bfabric.connect_webapp

Fixed

• References correctly handles references with extra fields like _position.

• ResultContainer.to_polars sets the schema length to None to fix bugs in some cases with more than 100 items.

[1.16.1] - 2025-12-15

Fixed

• Dataset correctly handles None values in items.

[1.16.0] - 2025-12-15

Added

• bfabric.typing with specific types for the API request and response objects.

• Executable.decoded_str, Executable.decoded_bytes

• Workunit.workunit_parameters to list workunit context parameters

• ExternalJob.client_entity

Changed

• Type hints have been narrowed in the public interfaces of the following classes:

  • Bfabric

  • ResultContainer

  • References

  • Entity

  • EntityReader

  • EngineSuds

  • EngineZeep

• Parameter.value returns "" for values not specified with required = false. This solves a few rare compatibility bugs and I do not think having the ability to distinguish between None and "" makes much sense at this level.

Removed

• Legacy workaround for B-Fabric with non-list technology field is removed. Assumed to not be breaking since for the current B-Fabric instances, this change should be transparent.

Deprecated

• Executable.decoded: misleading name

[1.15.1] - 2025-12-09

Fixed

• Allow http://localhost for EntityURIs.

[1.15.0] - 2025-12-09

Added

• TokenData.web_service_user which indicates whether the user has permission to use webservices API.

[1.14.1] - 2025-12-02

Added

• EntityReader.read_ methods accept a expected_type parameter to specify the expected type of the entity (default Entity).

Fixed

• A few imports within the package were accidentally broken in 1.14.0 which is resolved now. basedpyright has been added to identify issues like this earlier.

[1.14.0] - 2025-12-02

• The minimal Python version has been updated to 3.11.

• bfabric.entities

  • The read-only API in bfabric.entities has been redesigned to support multiple B-Fabric instances cleanly.

  • In particular, when handling configuration files it should now be obvious how to specify entities by their URI rather than having to specify instance, classname, and id separately.

  • The main new concept is the EntityUri which standardizes the way entities are specified by their URI.

  • EntityReader is a new class which implements the resolution, currently for only one B-Fabric instance, but extensible to support multiple B-Fabric instances in the future.

  • We also remove the reliance on custom entity classes providing generic functionality to read entities and their references. This is available through the EntityReader class which can be accessed through Bfabric.reader.

• Versioning:

  • The bfabric Python package versioning starting with 1.14.0 does not track the B-Fabric server version anymore.

  • This will allow us to adapt to implement a more semantically meaningful versioning, starting with 1.14.0.

  • Applications, like bfabric-scripts can instead track the particular version of bfabric against which they are tested/developed either in their version (planned for bfabric-scripts) or through documentation.

Added

• Entity.uri property to obtain the URI of an entity.

• Entity.refs resolves all references of entities generically, writes the data into data_dict and is compatible with pre-loaded references from B-Fabric.

• bfabric.entities.core.uri.EntityUri to identify entities regardless of B-Fabric instance.

• bfabric.entities.core.uri.EntityUriComponents to access individual components (bfabric_instance, entity_type, entity_id) from a URI.

• bfabric.entities.core.uri.GroupedUris helper for looping over lists of EntityUris.- bfabric.entities.core.entity_reader which allows reading entities by URI, ID, and general queries.

• bfabric.entities.cache which supersedes bfabric.experimental.cache.

• HasOne and HasMany should be able to use resolved entities when loading with fulldetails=True.

• Entity.custom_attributes for reading custom attribute values.

• bfabric.experimental.update_custom_attributes for updating custom attributes correctly.

Changed

• Minimal Python version is now 3.11.

• BFabricClientConfig does not allow passing None for values anymore.

• HasMany retains response order, instead of redundant sorting by ID.

• bfabric.entities allows loading entity references without custom definitions in Python.

• Internal

  • Entity has a new constructor parameter bfabric_instance which in the future will become mandatory.

  • BfabricClientConfig.base_url always ends with exactly one / now.

  • bfabric.entities do not define custom constructors anymore, simplifying future changes (and removing tiny inconsistencies).

  • TokenData retrieval uses async httpx internally, but provides a sync interface for compatibility.

Removed

• bfabric.experimental.cache is removed in favor of bfabric.entities.cache.

• Future deprecations:

  • Entity.find, Entity.find_all, Entity.find_by will be removed in favor of EntityReader methods.

  • Passing entity class to cache_context will be removed in favor of their name (this is because not all entities have custom classes).

[1.13.36] - 2025-10-27

Changed

• This is the last release supporting Python 3.9. From the next release onwards, Python 3.11 or higher will be required.

• Flask is not a dependency anymore, as the dependency will be moved to bfabric-scripts.

• Version upper bounds for dependencies have been defined.

• More robust URL handling for token authentication.

Fixed

• A test-only incompatibility with Polars was fixed.

[1.13.35] - 2025-09-22

Changed

• Dataset column type detection now supports case-insensitive entity matching (e.g., “resource” is detected as “Resource” type automatically).

[1.13.34] - 2025-09-19

Added

• bfabric.experimental.cache which implements re-entrant lookup caching for entities (when retrieved by ID).

• Fields filename, storage_relative_path, storage_absolute_path have been added to Resource. This should be used in the future, to ensure path handling is performed consistently

Removed

• bfabric.experimental.entity_lookup_cache has been removed in favor of the new (experimental) API.

[1.13.33] - 2025-08-26

Added

• Add new field WorkunitDefinition.registration.user_id.

• Add created/modified fields to Workunit entity.

Fixed

• Legacy wrapper creator also writes the inputdataset name.

[1.13.32] - 2025-08-26

Added

• New entities: Workflow, WorkflowStep, WorkflowTemplate, WorkflowTemplateStep

• New relationship: HasUser to resolve user references (since these are modeled differently from other relationships).

Fixed

• There was an issue in import_entity.py which made loading entities with additional uppercase characters in the type fail.

• Fix compatibility in legacy wrapper creator for dataset-flow workunits.

[1.13.31] - 2025-09-20

Changed

• The Bfabric.connect_webapp will use the caller field from the token data to set the base_url of the client.

  • Existing code should not break, but will emit a deprecation warning as we plan to remove the old parameters.

  • Theoretically, it changes the semantics, but in practice it should yield the same result for all known use cases.

Added

• TokenData will include the caller url now.

• Entity User has been added.

Fixed

• Legacy wrapper creator creates "" rather than None for empty parameter values.

[1.13.30] - 2025-08-19

Fixed

• Fix legacy wrapper creator for orders without projects.

[1.13.29] - 2025-07-04

Changed

• bfabric.rest.token_data.get_token_data now only requires the base_url instead of the whole config.

[1.13.28] - 2025-06-27

Breaking

• base_url is now a mandatory parameter in the configuration.

Added

• It is now possible to configure ~/.bfabricpy.yml without a default environment. In that case it will always be necessary to specify the requested config environment to be used.

• bfabric.entities.Executable has parameters relationship now

Removed

• Some old submitter related functionality is deleted.

Changed

• Columns of tables named after B-Fabric entities, containing only integers, will be set as the specified type when saving to B-Fabric (in experimental.upload_dataset).

[1.13.27] - 2025-05-21

Added

• Attribute Bfabric.config_data to obtain a ConfigData object directly.

• TokenData.load_entity convenience method to load an entity from the token data.

• Entities Instrument, Plate, Run were added (but with no extra functionality).

• Workunit.{application_parameters, submitter_parameters} to access parameter values.

Changed

• Submitter parameters will not be written any longer to WorkunitExecution parameters.

Fixed

• Compatibility with upcoming change that Application can have multiple technology values.

Deprecated

• Workunit.parameter_values will be removed in favor of Workunit.application_parameters and Workunit.submitter_parameters in a future version.

[1.13.26] - 2025-04-26

This release introduces an environment variable BFABRICPY_CONFIG_OVERRIDE to configure the Bfabric client completely, along with a new method for creating an instance of the Bfabric client, Bfabric.connect(). This will allow us to propagate any configuration to subprocesses reliably. BFABRICPY_CONFIG_ENV remains available with the same semantics, but lower priority than BFABRICPY_CONFIG_OVERRIDE.

This also simplifies the logic that was present in Bfabric.from_config which is why this introduced with a new API to prevent configuration mix-ups.

Added

• New environment variable BFABRICPY_CONFIG_OVERRIDE to configure the Bfabric client completely

• New method Bfabric.connect() for creating an instance of the Bfabric client

Changed

• Renamed Bfabric.from_token to Bfabric.connect_webapp() (along with some changes, no known users of this API yet)

• Disallowed default as an environment config name

• bfabric.cli_integration.utils.use_client uses Bfabric.connect() instead of Bfabric.from_config()

Deprecated

• Bfabric.from_config is now deprecated in favor of Bfabric.connect()

[1.13.25] - 2025-04-22

Breaking

• Bfabric.from_token returns the TokenData in addition to the client instance. While this is breaking, I’m not aware of any existing users of this API and I noticed that this information is going to be needed in this context often.

Added

• bfabric.experimental.upload_dataset.warn_on_trailing_spaces function used by bfabric-scripts to validate

[1.13.24] - 2025-04-08

Removed

• cyclopts is not a dependency of bfabric anymore, but rather of bfabric-scripts and bfabric-app-runner.

Changed

• Bfabric client now does not take engine argument anymore, but rather this information comes from ClientConfig. For compatibility reasons, the engine argument is still accepted in Bfabric.from_config and Bfabric.from_token.

[1.13.23] - 2025-03-25

Fixed

• Unsuccessful deletions are detected by checking the B-Fabric response.

• Handle problematic characters in Workunit.store_output_folder.

• BfabricRequestError did not properly subclass RuntimeError.

Changed

• WorkunitDefinition uses PathSafeStr to normalize app and workunit names.

• Internal: ResultContainer has no optional constructor arguments anymore to avoid confusion.

Added

• Generic functionality in bfabric.utils.path_safe_name to validate names for use in paths.

• bfabric.entities.Dataset.{write_parquet, get_parquet} methods for writing parquet

[1.13.22] - 2025-02-19

Fixed

• Correctly read datasets, if columns were swapped in B-Fabric.

Changed

• Dataset.types was renamed to Dataset.column_types, since there is only 1 usage and it is recent, this is not considered breaking.

[1.13.21] - 2025-02-19

Added

• Entity.load_yaml and Entity.dump_yaml

• Bfabric.from_token to create a Bfabric instance from a token

• bfabric.rest.token_data to get token data from the REST API, low-level functionality

Changed

• Internally, the user password is now in a pydantic.SecretStr until we construct the API call. This should prevent some logging related accidents.

[1.13.20] - 2025-02-10

Breaking

• the old bfabric.cli_formatting and bfabric.bfabric2 modules have been deleted

Changed

• move use_client to bfabric.utils.cli_integration: this will allow reuse between bfabric-scripts and bfabric-app-runner

• move functionality from bfabric.cli_formatting to bfabric.utils.cli_integration

[1.13.19] - 2025-02-06

Fixed

• Config: Log messages of app runner are shown by default again.

[1.13.18] - 2025-01-28

Changed

• bfabric_upload_resource.py does not print a list anymore, but rather only the dict of the uploaded resource.

Fixed

• Some scripts were not Python 3.9 compatible, which is restored now. However, we can consider increasing the Python requirement soon.

[1.13.17] - 2025-01-23

Added

• bfabric-cli workunit export-definition to export workunit_definition.yml files

• Executable.storage relationship

Fixed

• Workunit.parameters and Workunit.resources are optional

• bfabric-cli had unmet dependencies, that were not caught by the tests either

[1.13.16] - 2025-01-22

Added

• Add missing Entity.__contains__ implementation to check if a key is present in an entity.

• polars_utils.py which contains functionality to normalize relational fields in tables

• Add bfabric-cli executable inspect command to inspect executables registered in B-Fabric.

• Add bfabric-cli executable upload command to upload executables to B-Fabric.

Fixed

• Order.project is optional

[1.13.15] - 2025-01-15

Added

• Entity Sample

• Relationship Resource.sample.

• Extract logic for container resolution into HasContainerMixin which is right now shared between Sample and Workunit.

Fixed

• Fix bug in bfabric_save_fasta.py.

[1.13.14] - 2025-01-08

Changed

• Slurm submitter prints more diagnostics about PATH variable etc.

Fixed

• Some commands in bfabric-cli are broken because __future__.annotations is imported and this breaks cyclopts.

[1.13.13] - 2024-12-18

Changed

• Move bfabric-cli read to bfabric-cli api read.

• (internal) use_client decorator is introduced to simplify and standardize the client usage in the new CLI code.

Added

• Functionality to log by specifying the workunit instead of external job, which is used in some legacy scripts.

• Add logging commands to bfabric-cli api log.

• Add bfabric-cli api save.

• Submitter script prints the id and hostname of apps, which in practice can be very useful.

Fixed

• A bug in bfabric_save_workflowstep.py that crashed the script.

[1.13.12] - 2024-12-17

Changed

• The submitter ensures that workunits always get set to processing.

Fixed

• The script bfabric_setResourceStatus_available.py and other uses of report_resource, correctly search files which have a relative path starting with / as in the case of the legacy wrapper creator.

[1.13.11] - 2024-12-13

Fixed

• The script bfabric_setWorkunitStatus.py did always set the status to available, instead of the specified status.

Changed

• bfabric_setResourceStatus_available.py calls the report_resource function, in general this functionality has been refactored.

[1.13.10] - 2024-12-13

Fixed

• If bfabricpy.yml contains a root-level key which is not a dictionary, a correct error message is shown instead of raising an exception.

• A bug introduced while refactoring in slurm.py which passed Path objects to subprocess instead of strings.

• Submitter

  • does not use unset JOB_ID environment variable anymore.

  • does not set unused STAMP environment variable anymore.

• Fix bfabric_save_workflowstep.py bugs from refactoring.

Added

• Experimental bfabric-cli interface. Please do not use it in production yet as it will need a lot of refinement.

[1.13.9] - 2024-12-10

From this release onwards, the experimental app runner is not part of the main bfabric package and instead a separate Python package with its individual changelog.

Added

• Relationship: ExternalJob.executable

• (experimental) EntityLookupCache that allows to cache entity lookups in a script to avoid redundant requests.

• Specific use case script: bfabric_save_resource_description.py (the functionality will be available in a future CLI).

Fixed

• Entity.find_all returns no values when an empty list is passed as an argument.

Changed

• Except for macOS x86_64 (which we assume is Rosetta emulation nowadays), we use the faster polars instead of polars-lts-cpu.

• BfabricRequestError is now a RuntimeError subclass.

• Add py.typed marker.

Removed

• bfabric_legacy.py has been removed.

• math_helper.py has been removed.

[1.13.8] - 2024-10-03

This release contains mainly internal changes and ongoing development on the experimental app interface functionality.

Added

• Entities can be compared and sorted by ID now.

• Show Python version in version info string.

• Caching for bfabric_list_not_existing_storage_directories.py.

• (experimental) add initial code for a resource based application dispatch

• (experimental) new app_runner cli that integrates all commands into a single interface

Fixed

• bfabric_read.py is a bit more robust if “name” misses and tabular output is requested.

Changed

• bfabric.scripts has been moved into a namespace package bfabric_scripts so we can later split it off.

• (internal) migrate to src layout

• (experimental) the former process step of the app runner has been split into a process and collect step where, the collect step is responsible for generating the output.yml file that will then be used to register the results.

• (experimental) app runner apps by default reuse the default resource

[1.13.7] - 2024-09-17

Fixed

• bfabric_save_csv2dataset.py considers all rows for schema inference, only considering the first 100 rows did cause problems with some files previously.

Added

• Script logging configuration

  • If BFABRICPY_DEBUG environment variable is set, log messages in scripts will be set to debug mode.

  • Log messages from __main__ will also be shown by default.

• bfabric.entities.MultiplexKit to extract multiplex kit information.

• bfabric.entities.Workunit.store_output_folder implements the old rule, but more deterministically and reusable.

• (Experimental) bfabric.experimental.app_interface core functionality is implemented

  • This can be used as a building block to standardize the input preparation for applications.

Changed

• Correctly support optional workunit parameters.

Removed

• bfabric.entities.Resource association application has been removed as it does not exist

[1.13.6] - 2024-08-29

Added

• Entity.find_by has new parameter max_results.

Changed

• bfabric_read.py prints non-output information exclusively through logger and does not restrict entity types anymore.

• bfabric_delete.py accepts multiple ids at once and does not restrict entity types anymore.

• Both engines raise a BfabricRequestError if the endpoint was not found.

Removed

• bfabric.endpoints list is deleted, since it was out of date and generally not so useful.

[1.13.5] - 2024-08-13

Added

• The Bfabric instance is now pickleable.

• Entities mapping:

  • Add Entity.id and Entity.web_url properties.

  • Add Entity.__getitem__ and Entity.get to access fields from the data dictionary directly.

  • Add Entity.find_by to find entities by a query.

  • More types and relationships

  • Relationships defer imports to descriptor call, i.e. circular relationships are possible now.

  • HasOne and HasMany allow defining optional=True to indicate fields which can be missing under some circumstances.

• Add nodelist column and application name to bfabric_list_not_available_proteomics_workunits.py output.

Changed

• Entity.find_all supports more than 100 IDs now by using the experimental MultiQuery API.

[1.13.4] - 2024-08-05

Added

• Add Workunit, Parameter, and Resource entities.

• Add concept of has_many and has_one relationships to entities.

• bfabric_slurm_queue_status.py to quickly check slurm queue status.

• Bfabric.save provides method which can be set to checkandinsert for specific use cases.

Changed

• Most messages are now logged to debug level.

• The old verbose version information is now always logged, to INFO level, since it could entail useful information for error reporting.

[1.13.3] - 2024-07-18

Added

• Flask

  • New endpoint GET /config/remote_base_url for testing

Changed

• Flask

  • Simplify logging by using loguru only.

  • Simplified setup logic since the production use case should use a WSGI server.

Fixed

• bfabric_save_csv2dataset.py had an undeclared dependency on numpy and a few bugs which was improved.

[1.13.2] - 2024-07-11

Added

• Add bfabric.entities.Dataset to easily read datasets.

• Pydantic-based configuration parsing

  • The config format did not change.

  • The code is easier to maintain now.

  • Additionally, there is a lot more validation of the configuration file now, that should catch errors early.

• Make host and port configurable in bfabric_flask.py (currently only dev mode).

[1.13.1] - 2024-07-02

Changed

• bfabric_save_csv2dataset will raise an error if problematic characters are found in any of the cells

• Correctly define bfabric_setWorkunitStatus_available.py, and processing and failed variants.

Added

• Add loguru for future logging refactoring.

• Easily runnable tests with nox and standardized formatting using pre-commit.

Removed

• Pandas is no longer a dependency, and has been replaced by polars.

[1.13.0] - 2024-05-24

This is a major release refactoring bfabricPy’s API.

Changed

• The Bfabric class operations now return ResultContainer objects.

  • These provide a list-like interface to access individual items or iterate over them.

  • Individual items are a dictionary, potentially nested, and not specific to suds/zeep anymore.

  • Convenience conversions, e.g. to a polars DataFrame, can be provided there.

• Configuration is now defined in ~/.bfabricpy.yml and supports multiple configurations, which can be selected by the BFABRICPY_CONFIG_ENV environment variable. Please consult the README for an example configuration.

• Use pyproject.toml for package configuration.

• Scripts have been refactored on a case-by-case basis.

Added

• Zeep can be used instead of suds for SOAP communication.

• Bfabric can be instantiated without authentication, that can be provided later. This is useful in a server setup.

• Pagination support in Bfabric, specify the number of max_results and a potential offset. Pages handling is abstracted away.

• Detect errors in responses, e.g. invalid login.

Removed

• Several old scripts have been moved into a deprecated_scripts folder.

• Wrapper creator related code is currently not updated but has been extracted into a dedicated folder wrapper_creator as well.