AiiDA v2.9.0 released

We are happy to announce the release of AiiDA v2.9.0! The headline feature of this minor release is the new built-in ZeroMQ broker, which lets you submit calculations and workflows to the daemon without installing and running an external message broker such as RabbitMQ. It also adds a verdi bug-report command, an overhauled logging system, a Pydantic-based ORM model system, and support for dedicated data transfer nodes in the async SSH transport.

As always, the release can be installed from PyPI using pip install aiida-core==2.9.0, or from conda-forge using conda install -c conda-forge aiida-core=2.9.0. For a complete list of changes, see the CHANGELOG.

Behavior changes: This release includes several behavior changes that do not constitute API breaking changes warranting a major version bump, but may still affect existing workflows if you rely on the old defaults. Please review the following list and check the detailed sections below:

• Python 3.9 support dropped; the minimum supported version is now 3.10
• verdi presto falls back to the ZeroMQ broker instead of creating a brokerless profile when RabbitMQ is not reachable
• verdi storage maintain now cleans up loose files incrementally while packing
• logging.db_loglevel renamed to logging.database_handler
• The aiida logger no longer propagates to the root logger
• ORM entities expose specialized model classes; Entity.Model is deprecated
• The configuration file schema was bumped to version 10

Below we highlight the main new features.

ZeroMQ broker: run AiiDA without external services

AiiDA needs a message broker to talk to its daemon workers, and until now RabbitMQ was the only option. Installing and managing it separately complicates the setup, especially on machines where you do not have administrator rights, such as HPC login nodes.

This release adds a second broker plugin, core.zeromq, next to the existing core.rabbitmq (#7340). It requires no external service: the broker is started and supervised automatically by the daemon. Full process control, so submit(), verdi process play/pause/kill and verdi daemon start/stop, now works out of the box with nothing but pip install aiida-core.

verdi presto first checks whether RabbitMQ is running on localhost. If it can connect, the profile is configured with RabbitMQ as before; otherwise it now falls back to the ZeroMQ broker rather than creating a brokerless profile. Use verdi presto --no-broker if you explicitly want a profile without any broker. The broker can also be chosen at profile creation with verdi profile setup --broker, and existing profiles can be switched with the new verdi profile configure-broker command:

$ verdi profile configure-broker core.rabbitmq

For high-throughput production workloads RabbitMQ remains the recommended broker. The ZeroMQ broker is ideal for getting started, for light production use, and for machines where installing services is not an option. See the installation guide for the updated database and broker compatibility matrix, and the new broker internals documentation for the architecture.

verdi bug-report: bundle diagnostics for bug reports

Asking for help usually starts with a round of questions about versions, profile configuration and log files. The new verdi bug-report command collects all of that into a single zip archive:

• AiiDA and Python versions, installed packages, and platform details
• profile configuration and the AiiDA configuration file
• service status for storage, broker, and daemon
• the tails of the profile, daemon, circus, and broker log files

Secrets such as passwords are redacted before the configuration is embedded, but do review the archive before posting it publicly. Attach it when asking for help on Discourse or when opening an issue on GitHub.

Logging system overhaul

The logging configuration has been reworked to be easier to understand and control (#7322).

Output from verdi commands and from processes run outside the daemon is now also written to a per-profile log file, so each profile has three log files: the client-side log, the daemon worker log, and the circus supervisor log. The log files capture all messages, and two new options control what is shown elsewhere: logging.terminal_handler sets the minimum level for messages printed to the terminal, and logging.database_handler sets the minimum level for messages stored on a node, which is what verdi process report displays.

The per-library options such as logging.sqlalchemy_loglevel are now marked as advanced and hidden from verdi config list unless you pass --advanced, and levels for aiidateam packages inherit from logging.aiida_loglevel unless set explicitly. AiiDA also warns when a logging configuration has no effect because no handler logs at that level. See the updated troubleshooting documentation for details.

ORM model system

AiiDA’s ORM entities now expose their schema through a Pydantic-based model system, which makes it possible to introspect entities programmatically and to serialize them to and from formats such as JSON (#6990). Instead of a single Model, each entity provides a schema per context: ReadModel for read and serialization schemas, WriteModel for attribute-based schemas, and, where supported, ConstructorModel and CliModel.

The public Model attribute stays available as a deprecated compatibility wrapper, so existing plugins keep working while the model system stabilizes. New integrations should use the specialized classes directly.

Transport: dedicated data transfer nodes

Many HPC centers provide a dedicated data transfer node and ask users not to run heavy file transfers on the login node. The async SSH transport core.ssh_async now accepts a data_node_host option that routes SFTP and scp transfers to such a node, while calculations are still submitted and monitored on the login host (#7460). Both backends support it, and leaving the option unset keeps the previous behavior.

The openssh backend also gained a use_sftp option (#7423): since OpenSSH 9.0, scp transfers over SFTP by default, so configuring a computer with --no-use-sftp forces the legacy RCP protocol for servers without SFTP support. See the SSH how-to for details.

Smaller additions

• Schedulers can declare whether they support detailed job info via the new can_get_detailed_job_info property, which stops the spurious parsing warnings for schedulers such as core.direct (#7463, #7521).
• The new aiida.tools.WorkflowTools class, reachable as WorkChainNode.tools, bundles utilities for introspecting workflow state (#6884).
• DaemonClient.increase_workers() and decrease_workers() accept a wait parameter that blocks until the requested number of workers is active (#7502).

Configuration file: schema version 10

The configuration file schema was bumped to version 10, with a migration that renames the deprecated logging and broker options to their replacements. It is applied automatically the first time the configuration file is accessed. If you want to go back to an earlier release afterwards, v2.8.1 and v2.7.4 ship the matching downgrade migration, so you can switch back without editing config.json by hand.

Deprecations

• verdi profile setup --use-rabbitmq/--no-use-rabbitmq is deprecated in favor of --broker, and verdi profile configure-rabbitmq in favor of verdi profile configure-broker core.rabbitmq.
• The configuration options logging.db_loglevel and rmq.task_timeout are deprecated in favor of logging.database_handler and broker.task_timeout.
• Entity.Model, Entity.model_to_orm_fields(), Entity.model_to_orm_field_values() and several QbField attributes are deprecated as part of the new ORM model system.
• ArrayData.save_arrays() and ArrayData.load_arrays() are deprecated in favor of numpy.save and numpy.load with io.BytesIO.

Happy computing!

The AiiDA team