# Changelog The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). Versioning currently follows `X.Y.Z` semantic versioning. Historically, before 1.14.0, a different versioning scheme was used. Minor breaking changes are still possible in `1.X.Y` but we try to announce them with DeprecationWarnings in the previous `Y-1` release. ## \[Unreleased\] ## \[1.18.0\] - 2026-04-20 ### Added - `EntityReader.query_one`: returns at most one matching entity (or `None`). - `User.is_employee` property: `True` iff the user's `empdegree` parses to a value `> 0`. - `BfabricTokenValidationFailedError` in `bfabric.errors`: raised when token validation fails. - CLI commands decorated with `use_client` now accept `--config-env` and `--config-file` flags. ### Changed - `EntityReader.query` accepts an `expected_type` keyword for typed narrowing of results. - `User.find_by_login` no longer triggers a `FindMixin` deprecation warning. ### Fixed - `FindMixin.find_all` now correctly handles string IDs. ## \[1.17.0\] - 2026-03-12 ### Added - Improved token authentication - `bfabric.experimental.webapp_integration_settings` - `Bfabric.connect_token`, `Bfabric.connect_token_async`: this one respects the list of allowed bfabric instances - `bfabric.rest.token_data.validate_token` - `Bfabric.from_token_data`: creates a new Bfabric instance from token data. - `ResultContainer.to_polars` now has a `flatten` parameter to flatten struct columns into individual columns. ### Deprecated - `Bfabric.from_config` - `Bfabric.connect_webapp` ### Changed - `WorkunitExecutionDefinition` no longer validates that dataset or resources are provided. - `setup_script_logging()` now supports a `BFABRICPY_LOG_LEVEL` environment variable (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`, `OFF`/`0`). The previous `BFABRICPY_DEBUG` environment variable has been removed; use `BFABRICPY_LOG_LEVEL=DEBUG` instead. ### Fixed - `References` correctly handles references with extra fields like `_position`. - `ResultContainer.to_polars` sets the schema length to `None` to fix bugs in some cases with more than 100 items. ## \[1.16.2\] - 2026-03-02 ### Fixed - `BfabricAuth` does not prevent users with username shorter than 3 characters from logging in. ### Changed - `BfabricAPIEngineType` is a `StrEnum` now, rather than `str, Enum` subclass. ## \[1.16.1\] - 2025-12-15 ### Fixed - `Dataset` correctly handles `None` values in items. ## \[1.16.0\] - 2025-12-15 ### Added - `bfabric.typing` with specific types for the API request and response objects. - `Executable.decoded_str`, `Executable.decoded_bytes` - `Workunit.workunit_parameters` to list workunit context parameters - `ExternalJob.client_entity` ### Changed - Type hints have been narrowed in the public interfaces of the following classes: - `Bfabric` - `ResultContainer` - `References` - `Entity` - `EntityReader` - `EngineSuds` - `EngineZeep` - `Parameter.value` returns `""` for values not specified with `required` = `false`. This solves a few rare compatibility bugs and I do not think having the ability to distinguish between `None` and `""` makes much sense at this level. ### Removed - Legacy workaround for B-Fabric with non-list `technology` field is removed. Assumed to not be breaking since for the current B-Fabric instances, this change should be transparent. ### Deprecated - `Executable.decoded`: misleading name ## \[1.15.1\] - 2025-12-09 ### Fixed - Allow `http://localhost` for EntityURIs. ## \[1.15.0\] - 2025-12-09 ### Added - `TokenData.web_service_user` which indicates whether the user has permission to use webservices API. ## \[1.14.1\] - 2025-12-02 ### Added - `EntityReader.read_` methods accept a `expected_type` parameter to specify the expected type of the entity (default `Entity`). ### Fixed - A few imports within the package were accidentally broken in 1.14.0 which is resolved now. `basedpyright` has been added to identify issues like this earlier. ## \[1.14.0\] - 2025-12-02 - The minimal Python version has been updated to 3.11. - *bfabric.entities* - The read-only API in `bfabric.entities` has been redesigned to support multiple B-Fabric instances cleanly. - In particular, when handling configuration files it should now be obvious how to specify entities by their URI rather than having to specify instance, classname, and id separately. - The main new concept is the `EntityUri` which standardizes the way entities are specified by their URI. - `EntityReader` is a new class which implements the resolution, currently for only one B-Fabric instance, but extensible to support multiple B-Fabric instances in the future. - We also remove the reliance on custom entity classes providing generic functionality to read entities and their references. This is available through the `EntityReader` class which can be accessed through `Bfabric.reader`. - Versioning: - The `bfabric` Python package versioning starting with 1.14.0 does not track the B-Fabric server version anymore. - This will allow us to adapt to implement a more semantically meaningful versioning, starting with 1.14.0. - Applications, like `bfabric-scripts` can instead track the particular version of bfabric against which they are tested/developed either in their version (planned for `bfabric-scripts`) or through documentation. ### Added - `Entity.uri` property to obtain the URI of an entity. - `Entity.refs` resolves all references of entities generically, writes the data into `data_dict` and is compatible with pre-loaded references from B-Fabric. - `bfabric.entities.core.uri.EntityUri` to identify entities regardless of B-Fabric instance. - `bfabric.entities.core.uri.EntityUriComponents` to access individual components `(bfabric_instance, entity_type, entity_id)` from a URI. - `bfabric.entities.core.uri.GroupedUris` helper for looping over lists of `EntityUri`s.- `bfabric.entities.core.entity_reader` which allows reading entities by URI, ID, and general queries. - `bfabric.entities.cache` which supersedes `bfabric.experimental.cache`. - `HasOne` and `HasMany` should be able to use resolved entities when loading with `fulldetails=True`. - `Entity.custom_attributes` for reading custom attribute values. - `bfabric.experimental.update_custom_attributes` for updating custom attributes correctly. ### Changed - Minimal Python version is now 3.11. - `BFabricClientConfig` does not allow passing `None` for values anymore. - `HasMany` retains response order, instead of redundant sorting by ID. - `bfabric.entities` allows loading entity references without custom definitions in Python. - Internal - `Entity` has a new constructor parameter `bfabric_instance` which in the future will become mandatory. - `BfabricClientConfig.base_url` always ends with exactly one `/` now. - `bfabric.entities` do not define custom constructors anymore, simplifying future changes (and removing tiny inconsistencies). - `TokenData` retrieval uses async httpx internally, but provides a sync interface for compatibility. ### Removed - `bfabric.experimental.cache` is removed in favor of `bfabric.entities.cache`. - Future deprecations: - `Entity.find`, `Entity.find_all`, `Entity.find_by` will be removed in favor of `EntityReader` methods. - Passing entity class to `cache_context` will be removed in favor of their name (this is because not all entities have custom classes). ## \[1.13.36\] - 2025-10-27 ### Changed - This is the last release supporting Python 3.9. From the next release onwards, Python 3.11 or higher will be required. - Flask is not a dependency anymore, as the dependency will be moved to bfabric-scripts. - Version upper bounds for dependencies have been defined. - More robust URL handling for token authentication. ### Fixed - A test-only incompatibility with Polars was fixed. ## \[1.13.35\] - 2025-09-22 ### Changed - Dataset column type detection now supports case-insensitive entity matching (e.g., "resource" is detected as "Resource" type automatically). ## \[1.13.34\] - 2025-09-19 ### Added - `bfabric.experimental.cache` which implements re-entrant lookup caching for entities (when retrieved by ID). - Fields `filename`, `storage_relative_path`, `storage_absolute_path` have been added to `Resource`. This should be used in the future, to ensure path handling is performed consistently ### Removed - `bfabric.experimental.entity_lookup_cache` has been removed in favor of the new (experimental) API. ## \[1.13.33\] - 2025-08-26 ### Added - Add new field `WorkunitDefinition.registration.user_id`. - Add created/modified fields to `Workunit` entity. ### Fixed - Legacy wrapper creator also writes the inputdataset name. ## \[1.13.32\] - 2025-08-26 ### Added - New entities: `Workflow`, `WorkflowStep`, `WorkflowTemplate`, `WorkflowTemplateStep` - New relationship: `HasUser` to resolve user references (since these are modeled differently from other relationships). ### Fixed - There was an issue in `import_entity.py` which made loading entities with additional uppercase characters in the type fail. - Fix compatibility in legacy wrapper creator for dataset-flow workunits. ## \[1.13.31\] - 2025-09-20 ### Changed - The `Bfabric.connect_webapp` will use the `caller` field from the token data to set the `base_url` of the client. - Existing code should not break, but will emit a deprecation warning as we plan to remove the old parameters. - Theoretically, it changes the semantics, but in practice it should yield the same result for all known use cases. ### Added - `TokenData` will include the `caller` url now. - Entity `User` has been added. ### Fixed - Legacy wrapper creator creates `""` rather than `None` for empty parameter values. ## \[1.13.30\] - 2025-08-19 ### Fixed - Fix legacy wrapper creator for orders without projects. ## \[1.13.29\] - 2025-07-04 ### Changed - `bfabric.rest.token_data.get_token_data` now only requires the base_url instead of the whole config. ## \[1.13.28\] - 2025-06-27 ### Breaking - `base_url` is now a mandatory parameter in the configuration. ### Added - It is now possible to configure `~/.bfabricpy.yml` without a default environment. In that case it will always be necessary to specify the requested config environment to be used. - `bfabric.entities.Executable` has `parameters` relationship now ### Removed - Some old submitter related functionality is deleted. ### Changed - Columns of tables named after B-Fabric entities, containing only integers, will be set as the specified type when saving to B-Fabric (in `experimental.upload_dataset`). ## \[1.13.27\] - 2025-05-21 ### Added - Attribute `Bfabric.config_data` to obtain a `ConfigData` object directly. - `TokenData.load_entity` convenience method to load an entity from the token data. - Entities `Instrument`, `Plate`, `Run` were added (but with no extra functionality). - `Workunit.{application_parameters, submitter_parameters}` to access parameter values. ### Changed - Submitter parameters will not be written any longer to `WorkunitExecution` parameters. ### Fixed - Compatibility with upcoming change that `Application` can have multiple `technology` values. ### Deprecated - `Workunit.parameter_values` will be removed in favor of `Workunit.application_parameters` and `Workunit.submitter_parameters` in a future version. ## \[1.13.26\] - 2025-04-26 This release introduces an environment variable `BFABRICPY_CONFIG_OVERRIDE` to configure the `Bfabric` client completely, along with a new method for creating an instance of the `Bfabric` client, `Bfabric.connect()`. This will allow us to propagate any configuration to subprocesses reliably. `BFABRICPY_CONFIG_ENV` remains available with the same semantics, but lower priority than `BFABRICPY_CONFIG_OVERRIDE`. This also simplifies the logic that was present in `Bfabric.from_config` which is why this introduced with a new API to prevent configuration mix-ups. ### Added - New environment variable `BFABRICPY_CONFIG_OVERRIDE` to configure the `Bfabric` client completely - New method `Bfabric.connect()` for creating an instance of the `Bfabric` client ### Changed - Renamed `Bfabric.from_token` to `Bfabric.connect_webapp()` (along with some changes, no known users of this API yet) - Disallowed `default` as an environment config name - `bfabric.cli_integration.utils.use_client` uses `Bfabric.connect()` instead of `Bfabric.from_config()` ### Deprecated - `Bfabric.from_config` is now deprecated in favor of `Bfabric.connect()` ## \[1.13.25\] - 2025-04-22 ### Breaking - `Bfabric.from_token` returns the `TokenData` in addition to the client instance. While this is breaking, I'm not aware of any existing users of this API and I noticed that this information is going to be needed in this context often. ### Added - `bfabric.experimental.upload_dataset.warn_on_trailing_spaces` function used by `bfabric-scripts` to validate ## \[1.13.24\] - 2025-04-08 ### Removed - `cyclopts` is not a dependency of `bfabric` anymore, but rather of `bfabric-scripts` and `bfabric-app-runner`. ### Changed - `Bfabric` client now does not take `engine` argument anymore, but rather this information comes from `ClientConfig`. For compatibility reasons, the `engine` argument is still accepted in `Bfabric.from_config` and `Bfabric.from_token`. ## \[1.13.23\] - 2025-03-25 ### Fixed - Unsuccessful deletions are detected by checking the B-Fabric response. - Handle problematic characters in `Workunit.store_output_folder`. - `BfabricRequestError` did not properly subclass RuntimeError. ### Changed - `WorkunitDefinition` uses `PathSafeStr` to normalize app and workunit names. - Internal: `ResultContainer` has no optional constructor arguments anymore to avoid confusion. ### Added - Generic functionality in `bfabric.utils.path_safe_name` to validate names for use in paths. - `bfabric.entities.Dataset.{write_parquet, get_parquet}` methods for writing parquet ## \[1.13.22\] - 2025-02-19 ### Fixed - Correctly read datasets, if columns were swapped in B-Fabric. ### Changed - `Dataset.types` was renamed to `Dataset.column_types`, since there is only 1 usage and it is recent, this is not considered breaking. ## \[1.13.21\] - 2025-02-19 ### Added - `Entity.load_yaml` and `Entity.dump_yaml` - `Bfabric.from_token` to create a `Bfabric` instance from a token - `bfabric.rest.token_data` to get token data from the REST API, low-level functionality ### Changed - Internally, the user password is now in a `pydantic.SecretStr` until we construct the API call. This should prevent some logging related accidents. ## \[1.13.20\] - 2025-02-10 ### Breaking - the old `bfabric.cli_formatting` and `bfabric.bfabric2` modules have been deleted ### Changed - move `use_client` to `bfabric.utils.cli_integration`: this will allow reuse between bfabric-scripts and bfabric-app-runner - move functionality from `bfabric.cli_formatting` to `bfabric.utils.cli_integration` ## \[1.13.19\] - 2025-02-06 ### Fixed - Config: Log messages of app runner are shown by default again. ## \[1.13.18\] - 2025-01-28 ### Changed - `bfabric_upload_resource.py` does not print a list anymore, but rather only the dict of the uploaded resource. ### Fixed - Some scripts were not Python 3.9 compatible, which is restored now. However, we can consider increasing the Python requirement soon. ## \[1.13.17\] - 2025-01-23 ### Added - `bfabric-cli workunit export-definition` to export `workunit_definition.yml` files - `Executable.storage` relationship ### Fixed - `Workunit.parameters` and `Workunit.resources` are optional - `bfabric-cli` had unmet dependencies, that were not caught by the tests either ## \[1.13.16\] - 2025-01-22 ### Added - Add missing `Entity.__contains__` implementation to check if a key is present in an entity. - `polars_utils.py` which contains functionality to normalize relational fields in tables - Add `bfabric-cli executable inspect` command to inspect executables registered in B-Fabric. - Add `bfabric-cli executable upload` command to upload executables to B-Fabric. ### Fixed - `Order.project` is optional ## \[1.13.15\] - 2025-01-15 ### Added - Entity `Sample` - Relationship `Resource.sample`. - Extract logic for container resolution into `HasContainerMixin` which is right now shared between `Sample` and `Workunit`. ### Fixed - Fix bug in `bfabric_save_fasta.py`. ## \[1.13.14\] - 2025-01-08 ### Changed - Slurm submitter prints more diagnostics about PATH variable etc. ### Fixed - Some commands in bfabric-cli are broken because `__future__.annotations` is imported and this breaks cyclopts. ## \[1.13.13\] - 2024-12-18 ### Changed - Move `bfabric-cli read` to `bfabric-cli api read`. - (internal) `use_client` decorator is introduced to simplify and standardize the client usage in the new CLI code. ### Added - Functionality to log by specifying the workunit instead of external job, which is used in some legacy scripts. - Add logging commands to `bfabric-cli api log`. - Add `bfabric-cli api save`. - Submitter script prints the id and hostname of apps, which in practice can be very useful. ### Fixed - A bug in bfabric_save_workflowstep.py that crashed the script. ## \[1.13.12\] - 2024-12-17 ### Changed - The submitter ensures that workunits always get set to `processing`. ### Fixed - The script `bfabric_setResourceStatus_available.py` and other uses of `report_resource`, correctly search files which have a relative path starting with `/` as in the case of the legacy wrapper creator. ## \[1.13.11\] - 2024-12-13 ### Fixed - The script `bfabric_setWorkunitStatus.py` did always set the status to `available`, instead of the specified status. ### Changed - `bfabric_setResourceStatus_available.py` calls the `report_resource` function, in general this functionality has been refactored. ## \[1.13.10\] - 2024-12-13 ### Fixed - If `bfabricpy.yml` contains a root-level key which is not a dictionary, a correct error message is shown instead of raising an exception. - A bug introduced while refactoring in `slurm.py` which passed `Path` objects to `subprocess` instead of strings. - Submitter - does not use unset `JOB_ID` environment variable anymore. - does not set unused `STAMP` environment variable anymore. - Fix `bfabric_save_workflowstep.py` bugs from refactoring. ### Added - Experimental bfabric-cli interface. Please do not use it in production yet as it will need a lot of refinement. ## \[1.13.9\] - 2024-12-10 From this release onwards, the experimental app runner is not part of the main bfabric package and instead a separate Python package with its individual changelog. ### Added - Relationship: `ExternalJob.executable` - (experimental) EntityLookupCache that allows to cache entity lookups in a script to avoid redundant requests. - Specific use case script: bfabric_save_resource_description.py (the functionality will be available in a future CLI). ### Fixed - `Entity.find_all` returns no values when an empty list is passed as an argument. ### Changed - Except for macOS x86_64 (which we assume is Rosetta emulation nowadays), we use the faster `polars` instead of `polars-lts-cpu`. - `BfabricRequestError` is now a `RuntimeError` subclass. - Add `py.typed` marker. ### Removed - `bfabric_legacy.py` has been removed. - `math_helper.py` has been removed. ## \[1.13.8\] - 2024-10-03 This release contains mainly internal changes and ongoing development on the experimental app interface functionality. ### Added - Entities can be compared and sorted by ID now. - Show Python version in version info string. - Caching for bfabric_list_not_existing_storage_directories.py. - (experimental) add initial code for a resource based application dispatch - (experimental) new app_runner cli that integrates all commands into a single interface ### Fixed - bfabric_read.py is a bit more robust if "name" misses and tabular output is requested. ### Changed - `bfabric.scripts` has been moved into a namespace package `bfabric_scripts` so we can later split it off. - (internal) migrate to src layout - (experimental) the former `process` step of the app runner has been split into a `process` and `collect` step where, the collect step is responsible for generating the `output.yml` file that will then be used to register the results. - (experimental) app runner apps by default reuse the default resource ## \[1.13.7\] - 2024-09-17 ### Fixed - `bfabric_save_csv2dataset.py` considers all rows for schema inference, only considering the first 100 rows did cause problems with some files previously. ### Added - Script logging configuration - If `BFABRICPY_DEBUG` environment variable is set, log messages in scripts will be set to debug mode. - Log messages from `__main__` will also be shown by default. - `bfabric.entities.MultiplexKit` to extract multiplex kit information. - `bfabric.entities.Workunit.store_output_folder` implements the old rule, but more deterministically and reusable. - (Experimental) `bfabric.experimental.app_interface` core functionality is implemented - This can be used as a building block to standardize the input preparation for applications. ### Changed - Correctly support optional workunit parameters. ### Removed - `bfabric.entities.Resource` association `application` has been removed as it does not exist ## \[1.13.6\] - 2024-08-29 ### Added - `Entity.find_by` has new parameter `max_results`. ### Changed - `bfabric_read.py` prints non-output information exclusively through logger and does not restrict entity types anymore. - `bfabric_delete.py` accepts multiple ids at once and does not restrict entity types anymore. - Both engines raise a `BfabricRequestError` if the endpoint was not found. ### Removed - `bfabric.endpoints` list is deleted, since it was out of date and generally not so useful. ## \[1.13.5\] - 2024-08-13 ### Added - The `Bfabric` instance is now pickleable. - Entities mapping: - Add `Entity.id` and `Entity.web_url` properties. - Add `Entity.__getitem__` and `Entity.get` to access fields from the data dictionary directly. - Add `Entity.find_by` to find entities by a query. - More types and relationships - Relationships defer imports to descriptor call, i.e. circular relationships are possible now. - `HasOne` and `HasMany` allow defining `optional=True` to indicate fields which can be missing under some circumstances. - Add `nodelist` column and application name to `bfabric_list_not_available_proteomics_workunits.py` output. ### Changed - `Entity.find_all` supports more than 100 IDs now by using the experimental MultiQuery API. ## \[1.13.4\] - 2024-08-05 ### Added - Add `Workunit`, `Parameter`, and `Resource` entities. - Add concept of has_many and has_one relationships to entities. - `bfabric_slurm_queue_status.py` to quickly check slurm queue status. - `Bfabric.save` provides `method` which can be set to `checkandinsert` for specific use cases. ### Changed - Most messages are now logged to debug level. - The old verbose version information is now always logged, to INFO level, since it could entail useful information for error reporting. ## \[1.13.3\] - 2024-07-18 ### Added - Flask - New endpoint `GET /config/remote_base_url` for testing ### Changed - Flask - Simplify logging by using loguru only. - Simplified setup logic since the production use case should use a WSGI server. ### Fixed - `bfabric_save_csv2dataset.py` had an undeclared dependency on numpy and a few bugs which was improved. ## \[1.13.2\] - 2024-07-11 ### Added - Add `bfabric.entities.Dataset` to easily read datasets. - Pydantic-based configuration parsing - The config format did not change. - The code is easier to maintain now. - Additionally, there is a lot more validation of the configuration file now, that should catch errors early. - Make host and port configurable in `bfabric_flask.py` (currently only dev mode). ## \[1.13.1\] - 2024-07-02 ### Changed - bfabric_save_csv2dataset will raise an error if problematic characters are found in any of the cells - Correctly define `bfabric_setWorkunitStatus_available.py`, and `processing` and `failed` variants. ### Added - Add loguru for future logging refactoring. - Easily runnable tests with `nox` and standardized formatting using `pre-commit`. ### Removed - Pandas is no longer a dependency, and has been replaced by polars. ## \[1.13.0\] - 2024-05-24 This is a major release refactoring bfabricPy's API. ### Changed - The `Bfabric` class operations now return `ResultContainer` objects. - These provide a list-like interface to access individual items or iterate over them. - Individual items are a dictionary, potentially nested, and not specific to suds/zeep anymore. - Convenience conversions, e.g. to a polars DataFrame, can be provided there. - Configuration is now defined in `~/.bfabricpy.yml` and supports multiple configurations, which can be selected by the `BFABRICPY_CONFIG_ENV` environment variable. Please consult the README for an example configuration. - Use `pyproject.toml` for package configuration. - Scripts have been refactored on a case-by-case basis. ### Added - Zeep can be used instead of suds for SOAP communication. - `Bfabric` can be instantiated without authentication, that can be provided later. This is useful in a server setup. - Pagination support in `Bfabric`, specify the number of max_results and a potential offset. Pages handling is abstracted away. - Detect errors in responses, e.g. invalid login. ### Removed - Several old scripts have been moved into a `deprecated_scripts` folder. - Wrapper creator related code is currently not updated but has been extracted into a dedicated folder `wrapper_creator` as well.