virtmat.middleware package

Subpackages

Submodules

virtmat.middleware.exceptions module

specific exceptions for the vre-middleware

exception virtmat.middleware.exceptions.ConfigurationException

Bases: CustomException

Describes errors in the configuration of the function or missing parameters

exception virtmat.middleware.exceptions.CustomException

Bases: Exception

Custom exception class used for all specific exceptions

exception virtmat.middleware.exceptions.InvalidStateException(msg, node_id)

Bases: CustomException

Describes errors while being in a wrong state

exception virtmat.middleware.exceptions.ResourceConfigurationError

Bases: CustomException

Exception to raise from the resconfig module

exception virtmat.middleware.exceptions.SlurmError

Bases: RuntimeError

Exception to raise if Slurm is not responding or not available

exception virtmat.middleware.exceptions.TestingException

Bases: CustomException

Describes an exception while running a test

virtmat.middleware.utilities module

various utilities for the VRE middleware

virtmat.middleware.utilities.await_slurm_job_state(res_id, state, sleep_time=10)

return when a specified SLURM job state has been reached

virtmat.middleware.utilities.exec_cancel(res_id)

Execute the Slurm cancel command

Parameters:

res_id (int) – reservation ID (=Slurm JobID)

Raises:

SlurmError – if the command returns an error code

virtmat.middleware.utilities.format_warning_wrapper(func)

format user warnings, leave python warnings unchanged

virtmat.middleware.utilities.get_command_output(command)

run a slurm command (a string or list) and return the output

virtmat.middleware.utilities.get_logger(name, default_level=40)

return a custom logger object

virtmat.middleware.utilities.get_slurm_job_state(*args, method='json', **kwargs)

select the method to get the SLURM job state

virtmat.middleware.utilities.get_slurm_job_state_json(res_id, max_trials=3, trial_delay=3, trial=0)

extract the SLURM job state using the sacct command

virtmat.middleware.utilities.get_slurm_job_state_parse(res_id, max_trials=10, trial_delay=3, trial=0)

extract the SLURM job state using the sacct command

virtmat.middleware.utilities.get_unreserved_nodes(launchpad, fw_ids, expiration_time=1209600)

Detect reserved nodes, i.e. in ‘RESERVED’ state within FireWorks, that have not been updated for a while. Possible inconsistent states in SLURM are ‘CANCELLED’, ‘FAILED’, ‘COMPLETED’, ‘OUT_OF_MEMORY’, ‘BOOT_FAIL’, ‘TIMEOUT’ and ‘DEADLINE’

Parameters:

  • launchpad (LaunchPad) – launchpad object

  • fw_ids ([int]) – firework IDs

  • expiration_time (int) – minimum time in seconds since the most recent update, default is 1209600 seconds

Returns:

a list of dictionaries containing the firework ids, the reservation ids, the SLURM states and the launch directories of such reserved nodes

Return type:

([dict])

virtmat.middleware.utilities.lmod_env_module(command)

Process lmod modules list and spider commands and return modules

Due to ambiguous format of provided input it is assumed that the last division is the module version in the case of more than one division.

Parameters:

command (str) – either ‘list’ or ‘spider’

Returns:

{prefix/…/name: [version1, …]} in case of more divisions

{‘name’: []} in case of one division

Return type:

(dict)

Module contents