Model Drivers ¶

classmethod is_language_installed()[source]¶

Determine if the interpreter/compiler for the associated programming language is installed.

Returns:: True if the language interpreter/compiler is installed.
Return type:: bool

classmethod language_version(version_flags=None, **kwargs)[source]¶

Determine the version of this language.

Parameters:: **kwargs – Keyword arguments are passed to cls.run_executable.
Returns:: Version of compiler/interpreter for this language.
Return type:: str

class yggdrasil.drivers.PythonModelDriver.PythonModelDriver(*args, **kwargs)[source]¶

Class for running Python models.

static finalize_registration(cls)[source]¶: Operations that should be performed after a class has been fully initialized and registered.

classmethod format_function_param(key, default=None, **kwargs)[source]¶

Return the formatted version of the specified key.

Parameters:

key (str) – Key in cls.function_param mapping that should be formatted.
default (str, optional) – Format that should be returned if key is not in cls.function_param. Defaults to None.
**kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns:

Formatted string.

Return type:

Raises:

NotImplementedError – If key is not in cls.function_param and default is not set.

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

classmethod install_dependency(package=None, package_manager=None, **kwargs)[source]¶

Install a dependency.

Parameters:

package (str) – Name of the package that should be installed. If the package manager supports it, this can include version requirements.
package_manager (str, optional) – Package manager that should be used to install the package.
**kwargs – Additional keyword arguments are passed to the parent class.

classmethod is_language_installed()[source]¶

Determine if this model driver is installed on the current machine.

Returns:

Truth of if this model driver can be run on the current: machine.

Return type:

classmethod is_library_installed(lib, **kwargs)[source]¶

Determine if a dependency is installed.

Parameters:

lib (str) – Name of the library that should be checked.
**kwargs – Additional keyword arguments are ignored.

Returns:

True if the library is installed, False otherwise.

Return type:

run_validation()[source]¶: Run the validation script for the model.

set_env(**kwargs)[source]¶

Get environment variables that should be set for the model process.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method.
Returns:: Environment variables for the model process.
Return type:: dict

classmethod write_finalize_oiter(var)[source]¶

Get the lines necessary to finalize an array after iteration.

Parameters:: var (dict, str) – Name or information dictionary for the variable being initialized.
Returns:: The lines finalizing the variable.
Return type:: list

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]¶

Get the lines necessary to initialize an array for iteration output.

Parameters:

var (dict, str) – Name or information dictionary for the variable being initialized.
value (str, optional) – Value that should be assigned to the variable.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The lines initializing the variable.

Return type:

class yggdrasil.drivers.MatlabModelDriver.MatlabModelDriver(*args, **kwargs)[source]¶

Base class for running Matlab models.

Parameters:

name (str) – Driver name.
args (str or list) – Argument(s) for running the model in matlab. Generally, this should be the full path to a Matlab script.
use_symunit (bool, optional) – If True, input/output variables with units will be represented in Matlab using symunit. Defaults to False.
**kwargs – Additional keyword arguments are passed to parent class’s __init__ method.

started_matlab¶

True if the driver had to start a new matlab engine. False otherwise.

Type:: bool

screen_session¶

Screen session that Matlab was started in.

Type:: str

mlengine¶

Matlab engine used to run script.

Type:: object

mlsession¶

Name of the Matlab session that was started.

Type:: str

Raises:: RuntimeError – If Matlab is not installed.

Note

Matlab models that call exit will shut down the shared engine.

after_loop()[source]¶: Actions to perform after run_loop has finished. Mainly checking if there was an error and then handling it.

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

before_start()[source]¶: Actions to perform before the run loop.

check_exits()[source]¶

Check to make sure the program dosn’t contain any exits as exits will shut down the Matlab engine as well as the program.

Raises:: RuntimeError – If there are any exit calls in the file.

cleanup()[source]¶: Close the Matlab session and engine.

classmethod comm_atexit(comm)[source]¶

Operations performed on comm at exit including draining receive.

Parameters:: comm (CommBase) – Communication object.

classmethod configure(cfg, disable_engine=None, hide_matlab_libiomp=None)[source]¶

Add configuration options for this language. This includes locating any required external libraries and setting option defaults.

Parameters:

cfg (YggConfigParser) – Config class that options should be set for.
disable_engine (bool, optional) – Disable the use of the Python engine for Matlab. If not provided, this argument will be ignored.
hide_matlab_libiomp (bool, optional) – Hide the Matlab version of libiomp by moving it and recording the name of the file. If False, the file will be restored if it exists. If not provided, this argument will be ignored.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod decode_format(format_str)[source]¶

Method for decoding format strings created in this language.

Parameters:: format_str (str) – Encoded format string.
Returns:: Decoded format string.
Return type:: str

classmethod executable_command(args, **kwargs)[source]¶

Compose a command for running a program in this language with the provied arguments. If not already present, the interpreter command and interpreter flags are prepended to the provided arguments.

Parameters:

args (list) – The program that returned command should run and any arguments that should be provided to it.
**kwargs – Additional keyword arguments are ignored.

Returns:

Arguments composing the command required to run the program: from the command line using the interpreter for this language.

Return type:

classmethod get_matlab_info()[source]¶

Determine the root directory where Matlab is installed and the version that is installed (if Matlab is installed at all). This will fail if Matlab is not installed, cannot be started, or does not operate as expected.

Returns:: Matlab root directory and Matlab version string.
Return type:: tuple
Raises:: RuntimeError – If Matlab cannot be started or the root directory or release cannot be determiend.

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

classmethod language_version(skip_config=False)[source]¶

Determine the version of this language.

Parameters:: skip_config (bool, optional) – If True, the config option for the version (if it exists) will be ignored and the version will be determined fresh.
Returns:: Version of compiler/interpreter for this language.
Return type:: str

parse_arguments(args)[source]¶

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters:: args (list) – List of arguments provided.

classmethod run_code(lines, **kwargs)[source]¶

Run code by first writing it as an executable and then calling the driver.

Parameters:

lines (list) – Lines of code to be wrapped as an executable.
**kwargs – Additional keyword arguments are passed to the write_executable method.

classmethod run_executable(args, dont_wrap_error=False, fname_wrapper=None, matlab_engine=None, **kwargs)[source]¶

Run a program using the executable for this language and the provided arguments.

Parameters:

args (list) – The program that should be run and any arguments that should be provided to it.
dont_wrap_error (bool, optional) – If False, the executable will be wrapped in a try/catch block to prevent errors from stopping Matlab shutdown. If True, the command will be executed as is with the Matlab interpreter. Defaults to False.
fname_wrapper (str, optional) – File where wrapper should be saved. If not provided, one is created. Defaults to None.
matlab_engine (MatlabEngine, optional) – Matlab engine that should be used to run the command. If not provided, the Matlab interpreter is used instead. Defaults to None.
**kwargs – Additional keyword arguments are passed to cls.executable_command and tools.popen_nobuffer.

Returns:

Output to stdout from the run command.

Return type:

Raises:

RuntimeError – If the language is not installed.
RuntimeError – If there is an error when running the command.

run_loop()[source]¶: Loop to check if model is still running and forward output.

set_env()[source]¶

Get environment variables that should be set for the model process.

Returns:: Environment variables for the model process.
Return type:: dict

start_matlab_engine()[source]¶: Start matlab session and connect to it.

classmethod write_error_wrapper(fname, try_lines, env=None, matlab_engine=None)[source]¶

Write a wrapper for the model that encloses it in a try except so that the error can be propagated appropriately.

Parameters:

fname (str) – File where the wrapper should be written.
try_lines (list) – List of lines to go in the try block.
model_file (str) – Path to model that should be wrapped.
env (dict, optional) – Dictionary of environment variables that should be set before calling the model. Defaults to None and is ignored.
matlab_engine (MatlabEngine, optional) – Matlab engine that will be used to call the wrapper. If not provided, it is assumed the error will be called using the Matlab interpreter on the command line. Defautls to None.

Raises:

classmethod write_finalize_iiter(var, **kwargs)[source]¶

Get the lines necessary to initialize an array for iteration output.

Parameters:

var (dict, str) – Name or information dictionary for the variable being initialized.
value (str, optional) – Value that should be assigned to the variable.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The lines initializing the variable.

Return type:

classmethod write_finalize_oiter(var)[source]¶

Get the lines necessary to finalize an array after iteration.

Parameters:: var (dict, str) – Name or information dictionary for the variable being initialized.
Returns:: The lines finalizing the variable.
Return type:: list

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]¶

Get the lines necessary to initialize an array for iteration output.

Parameters:

var (dict, str) – Name or information dictionary for the variable being initialized.
value (str, optional) – Value that should be assigned to the variable.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The lines initializing the variable.

Return type:

class yggdrasil.drivers.MakeModelDriver.MakeModelDriver(*args, **kwargs)[source]¶

Class for running make file compiled drivers. Before running the make command, the necessary compiler & linker flags for the interface’s C/C++ library are stored the environment variables CFLAGS and LDFLAGS respectively. These should be used in the make file to correctly compile with the interface’s C/C++ libraries.

Parameters:

name (str) – Driver name.
args (str, list) – Executable that should be created (make target) and any arguments for the executable.
makefile (str, optional) – Path to make file either absolute, relative to makedir (if provided), or relative to working_dir. Defaults to Makefile.
makedir (str, optional) – Directory where make should be invoked from if it is not the same as the directory containing the makefile. Defaults to directory containing makefile if provided, otherwise working_dir.
**kwargs – Additional keyword arguments are passed to parent class.

makefile¶

Path to make file either relative to makedir or absolute.

Type:: str

makedir¶

Directory where make should be invoked from.

Type:: str

Raises:: RuntimeError – If neither the IPC or ZMQ C libraries are available.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

compile_model(target=None, **kwargs)[source]¶

Compile model executable(s).

Parameters:

target (str, optional) – Target to build.
**kwargs – Keyword arguments are passed on to the parent class’s method.

classmethod fix_path(path, for_env=False, **kwargs)[source]¶

Update a path.

Parameters:

path (str) – Path that should be formatted.
for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Updated path.

Return type:

classmethod get_buildfile_lock(**kwargs)[source]¶: Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

classmethod get_language_for_buildfile(buildfile, target=None)[source]¶

Determine the target language based on the contents of a build file.

Parameters:

buildfile (str) – Full path to the build configuration file.
target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_target_language_info(*args, **kwargs)[source]¶

Get a dictionary of information about language compilation tools.

Parameters:

*args – Arguments are passed to the parent class’s method.
**kwargs – Keyword arguments are passed to the parent class’s method.

Returns:

Information about language compilers and linkers.

Return type:

parse_arguments(args, **kwargs)[source]¶

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

class yggdrasil.drivers.CMakeModelDriver.CMakeModelDriver(*args, **kwargs)[source]¶

Class for running cmake compiled drivers. Before running the cmake command, the cmake commands for setting the necessary compiler & linker flags for the interface’s C/C++ library are written to a file called ‘ygg_cmake.txt’ that should be included in the CMakeLists.txt file (after the target executable has been added).

Parameters:

name (str) – Driver name.
args (str, list) – Executable that should be created (cmake target) and any arguments for the executable.
sourcedir (str, optional) – Source directory to call cmake on. If not provided it is set to working_dir. This should be the directory containing the CMakeLists.txt file. It can be relative to working_dir or absolute.
builddir (str, optional) – Directory where the build should be saved. Defaults to <sourcedir>/build. It can be relative to working_dir or absolute.
configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’.
**kwargs – Additional keyword arguments are passed to parent class.

sourcedir¶

Source directory to call cmake on.

Type:: str

add_libraries¶

If True, interface libraries and dependency libraries are added using CMake’s ADD_LIBRARY directive. If False, interface libraries are found using FIND_LIBRARY.

Type:: bool

configuration¶

Build type/configuration that should be built. This is only used on Windows.

Type:: str

Raises:: RuntimeError – If neither the IPC or ZMQ C libraries are available.

property buildfile_orig¶

Full path to where the original CMakeLists.txt file will be stored during compilation of the modified file.

Type:: str

property buildfile_ygg¶

Full path to the verison of the CMakeLists.txt that has been updated w/ yggdrasil compilation flags.

Type:: str

compile_model(target=None, **kwargs)[source]¶

Compile model executable(s) and appends any products produced by the compilation that should be removed after the run is complete.

Parameters:

target (str, optional) – Target to build.
**kwargs – Keyword arguments are passed on to the call_compiler method.

classmethod fix_path(path, for_env=False, **kwargs)[source]¶

Update a path.

Parameters:

path (str) – Path that should be formatted.
for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Updated path.

Return type:

classmethod get_language_for_buildfile(buildfile, target=None)[source]¶

Determine the target language based on the contents of a build file.

Parameters:

buildfile (str) – Full path to the build configuration file.
target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_target_language_info(target_compiler_flags=None, target_linker_flags=None, compiler_flag_kwargs=None, linker_flag_kwargs=None, without_wrapper=False, **kwargs)[source]¶

Get a dictionary of information about language compilation tools.

Parameters:

target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].
target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].
compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.
linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.
**kwargs – Keyword arguments are passed to the parent class’s method.

Returns:

Information about language compilers and linkers.

Return type:

parse_arguments(args, **kwargs)[source]¶

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod prune_sh_gcc(path, gcc)[source]¶

Remove instances of sh.exe from the path that are not associated with the selected gcc compiler. This can happen on windows when rtools or git install a version of sh.exe that is added to the path before the compiler.

Parameters:

path (str) – Contents of the path variable.
gcc (str) – Full path to the gcc executable.

Returns:

Modified path that removes the extra instances: of sh.exe.

Return type:

classmethod update_compiler_kwargs(**kwargs)[source]¶

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Keyword arguments for a get_flags method providing compiler: flags.

Return type:

write_wrappers(**kwargs)[source]¶

Write any wrappers needed to compile and/or run a model.

Parameters:: **kwargs – Keyword arguments are passed to the parent class’s method.
Returns:: Full paths to any created wrappers.
Return type:: list

class yggdrasil.drivers.LPyModelDriver.LPyModelDriver(*args, **kwargs)[source]¶

Class for running LPy models.

function_param = None¶

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

classmethod language_version(**kwargs)[source]¶

Determine the version of this language.

Parameters:: **kwargs – Keyword arguments are passed to cls.run_executable.
Returns:: Version of compiler/interpreter for this language.
Return type:: str

Model Base Classes¶

class yggdrasil.drivers.InterpretedModelDriver.InterpretedModelDriver(*args, **kwargs)[source]¶

Base class for models written in interpreted languages.

Parameters:

name (str) – Driver name.
args (str or list) – Argument(s) for running the model on the command line. This driver will check to see if there is an interpreter included in the args provided. If not, one will be added unless skip_interpreter is True.
interpreter (str, optional) – Name or path of interpreter executable that should be used to run the model. If not provided, the interpreter will be determined based on configuration options for the language (if present) and the default_interpreter class attribute.
interpreter_flags (list, optional) – Flags that should be passed to the interpreter when running the model. If not provided, the flags are determined based on configuration options for the language (if present) and the default_interpreter_flags class attribute.
skip_interpreter (bool, optional) – If True, no interpreter will be added to the arguments. This should only be used for subclasses that will not be invoking the model via the command line. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class.

Class Attributes:

default_interpreter (str): Name of interpreter that will be used if not: set explicitly by instance or config file. Defaults to the language name if not set.
default_interpreter_flags (list): Flags that will be passed to the: interpreter when running the model by default if not set explicitly by instance or config file.

interpreter¶

Name or path to the interpreter that will be used.

Type:: str

interpreter_flags¶

Flags that will be passed to the interpreter when running a model.

Type:: list

path_env_variable¶

Name of the environment variable containing path information for the interpreter for this language.

Type:: str

paths_to_add¶

Paths that should be added to the path_env_variable for this language on the process the model is run in.

Type:: list

comm_atexit¶

Function taking a comm instance as input that performs any necessary operations during exit. If None, no additional actions are taken.

Type:: function

comm_linger¶

If True, interface comms will linger during close. This should only be required if the language will disreguard Python threads at exit (e.g. when using a Matlab engine).

Type:: bool

decode_format (function: Function decoding format string created in this language. If None, no additional actions are taken.

recv_converters¶

Mapping between the names of message types (e.g. ‘array’, ‘pandas’) and functions that should be used to prepare such objects for return when they are received.

Type:: dict

send_converters¶

Mapping between the names of message types (e.g. ‘array’, ‘pandas’) and functions that should be used to prepare such objects for sending.

Type:: dict

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

classmethod are_dependencies_installed(**kwargs)[source]¶

Determine if the dependencies are installed for the interface (not including dependencies needed by a particular communication type).

Returns:: True if the dependencies are installed. False otherwise.
Return type:: bool

classmethod executable_command(args, exec_type='interpreter', unused_kwargs=None, skip_interpreter_flags=False, interpreter=None, interpreter_flags=None, **kwargs)[source]¶

Compose a command for running a program in this language with the provied arguments. If not already present, the interpreter command and interpreter flags are prepended to the provided arguments.

Parameters:

args (list) – The program that returned command should run and any arguments that should be provided to it.
exec_type (str, optional) – Type of executable command that will be returned. If ‘interpreter’, a command using the interpreter is returned and if ‘direct’, the raw args being provided are returned. Defaults to ‘interpreter’.
skip_interpreter_flags (bool, optional) – If True, interpreter flags will not be added to the command after the interpreter. Defaults to False. Interpreter flags will not be added, reguardless of this keyword, if the first element of args is already an interpreter.
unused_kwargs (dict, optional) – Existing dictionary that unused keyword arguments should be added to. Defaults to {}.
**kwargs – Additional keyword arguments are ignored.

Returns:

Arguments composing the command required to run the program: from the command line using the interpreter for this language.

Return type:

Raises:

ValueError – If exec_type is not ‘interpreter’ or ‘direct’.

classmethod get_interpreter()[source]¶

Command required to run a model written in this language from the command line.

Returns:: Name of (or path to) interpreter executable.
Return type:: str

classmethod get_interpreter_flags()[source]¶

Get the flags that should be passed to the interpreter when using it to run a model on the command line.

Returns:

Flags that should be passed to the interpreter on the command: line.

Return type:

classmethod is_interpreter(cmd)[source]¶

Determine if a command line argument is an interpreter.

Parameters:: cmd (str) – Command that should be checked.
Returns:: True if the command is an interpreter, False otherwise.
Return type:: bool

classmethod language2python(pyobj)[source]¶

Prepare an object from the target language for receipt in Python.

Parameters:

pyobj (object) – Python object transformed from the target language.

Returns:

Python object in a form that conforms with the: expected Python type.

Return type:

object

classmethod language_executable()[source]¶

Command required to compile/run a model written in this language from the command line.

Returns:

Name of (or path to) compiler/interpreter executable required: to run the compiler/interpreter from the command line.

Return type:

parse_arguments(*args, **kwargs)[source]¶

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters:

*args – Arguments are passed to the parent class’s method.
**kwargs – Keyword arguments are passed to the parent class’s method.

classmethod python2language(pyobj)[source]¶

Prepare a python object for transformation in target language.

Parameters:

pyobj (object) – Python object.

Returns:

Python object in a form that is friendly to the: target language.

Return type:

object

run_model(*args, **kwargs)[source]¶

Run the model. Unless overridden, the model will be run using run_executable.

Parameters:

*args – Arguments are passed to the parent class’s method.
**kwargs – Keyword arguments are passed to the parent class’s method.

set_env(**kwargs)[source]¶

Get environment variables that should be set for the model process.

Returns:: Environment variables for the model process.
Return type:: dict

class yggdrasil.drivers.CompiledModelDriver.CompiledModelDriver(*args, **kwargs)[source]¶

Base class for models written in compiled languages.

Parameters:

name (str) – Driver name.
args (str or list) – The model executable and any arguments that should be passed to the model executable.
source_files (list, optional) – Source files that should be compiled into an executable. Defaults to an empty list and the driver will search for a source file based on the model executable (the first model argument).
compiler (str, optional) – Command or path to executable that should be used to compile the model. If not provided, the compiler will be determined based on configuration options for the language (if present) and the registered compilers that are available on the current operating system.
compiler_flags (list, optional) – Flags that should be passed to the compiler during compilation. If nto provided, the compiler flags will be determined based on configuration options for the language (if present), the compiler defaults, and the default_compiler_flags class attribute.
linker (str, optional) – Command or path to executable that should be used to link the model. If not provided, the linker will be determined based on configuration options for the language (if present) and the registered linkers that are available on the current operating system
linker_flags (list, optional) – Flags that should be passed to the linker during compilation. If not provided, the linker flags will be determined based on configuration options for the language (if present), the linker defaults, and the default_linker_flags class attribute.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).
**kwargs – Additional keyword arguments are passed to parent class.

Class Attributes:

default_compiler (str): Name of compiler that will be used if not: set explictly by instance or config file.
default_compiler_flags (list): Flags that should be passed to the: compiler by default for this language.
default_linker (str): Name of linker that will be used if not: set explictly by instance or config file
default_linker_flags (list): Flags that should be passed to the: linker by default for this language.
allow_parallel_build (bool): If True, a file can be compiled by: two processes simultaneously. If False, it cannot and an MPI barrier will be used to prevent simultaneous compilation. Defaults to False.

source_files¶

Source files.

Type:: list

compiler¶

Name or path to the compiler that should be used.

Type:: str

compiler_flags¶

Compiler flags.

Type:: list

linker¶

Name or path to the linker that should be used.

Type:: str

linker_flags¶

Linker flags.

Type:: list

compiler_tool¶

Compiler tool that will be used.

Type:: CompilerBase

linker_tool¶

Linker tool that will be used.

Type:: LinkerBase

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

buildfile_locked(dry_run=False)[source]¶: Context manager for locked build file.

classmethod call_compiler(src, language=None, toolname=None, dont_build=None, **kwargs)[source]¶

Compile a source file into an executable or linkable object file, checking for errors.

Parameters:

src (str) – Full path to source file.
out (str, optional) – Full path to the output object file that should be created. Defaults to None and is created from the provided source file.
flags (list, optional) – Compilation flags. Defaults to []. If compiler_flags is present, flags is replaced by compiler_flags.
compiler_flags (list, optional) – Alternative to flags. Ignored if not provided.
dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.
dont_build (bool, optional) – If True, cmake configuration/generation will be run, but the project will not be built. Defaults to None. If provided, this overrides dont_link.
overwrite (bool, optional) – If True, the existing compile file will be overwritten. Otherwise, it will be kept and this function will return without recompiling the source file.
language (str, optional) – Language that should be used to compile the files. Defaults to None and the language of the current driver is used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
products (list, optional) – Existing Python list that additional products produced by the compilation should be appended to. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are passed to run_executable. and call_linker if dont_link is False.

Returns:

Full path to compiled source.

Return type:

Raises:

RuntimeError – If there is an error in calling the compiler.
RuntimeError – If the compilation command dosn’t yield the specified output file.

classmethod call_linker(obj, language=None, toolname=None, **kwargs)[source]¶

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters:

obj (list) – Object files that should be linked.
language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to run_executable.

Returns:

Full path to compiled source.

Return type:

classmethod cleanup_dependencies(products=None, verbose=False, **kwargs)[source]¶: Cleanup dependencies.

classmethod compile_dependencies(toolname=None, dep=None, **kwargs)[source]¶: Compile any required internal libraries, including the interface.

compile_dependencies_instance(*args, **kwargs)[source]¶: Compile dependencies specifically for this instance.

compile_model(source_files=None, skip_interface_flags=False, **kwargs)[source]¶

Compile model executable(s).

Parameters:

source_files (list, optional) – Source files that will be compiled. Defaults to None and is set to the source_files attribute.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. This includes the logger flag specifying the current logging level. Defaults to False.
**kwargs – Keyword arguments are passed on to the call_compiler method.

Returns:

Compiled model file path.

Return type:

classmethod configuration_steps()[source]¶

Get a list of configuration steps with tuples of flags and boolean values.

Returns:

Pairs of descriptions and states for: different steps in the configuration all steps must be True for the language to be configured.

Return type:

OrderedDict

classmethod configure(cfg, **kwargs)[source]¶

Add configuration options for this language.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.
**kwargs – Additional keyword arguments are used to set tool configuration options (e.g. ‘compiler’).

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_executable_type(cfg)[source]¶

Add configuration options specific in the executable type before the libraries are configured.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_libraries(cfg)[source]¶

Add configuration options for external libraries in this language.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_library(cfg, k)[source]¶

Add configuration options for an external library.

Parameters:

cfg (YggConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod executable_command(args, exec_type='compiler', toolname=None, **kwargs)[source]¶

Compose a command for running a program using the compiler for this language and the provied arguments. If not already present, the compiler command and compiler flags are prepended to the provided arguments.

Parameters:

args (list) – The program that returned command should run and any arguments that should be provided to it. For the compiler, this means the source files, for the linker, this means the object files.
exec_type (str, optional) – Type of executable command that will be returned. If ‘compiler’, a command using the compiler is returned, if ‘linker’, a command using the linker is returned, and if ‘direct’, the raw args being provided are returned. Defaults to ‘compiler’.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to either get_linker_flags or get_compiler_flags.

Returns:

Arguments composing the command required to run the program: from the command line using the compiler for this language.

Return type:

Raises:

ValueError – If exec_type is not ‘compiler’, ‘linker’, or ‘direct’.

classmethod get_available_tools(tooltype)[source]¶

Return the registry of compilation tools for this language.

Parameters:: tooltype (str) – Name of tool type that should be returned. Values include ‘compiler’, ‘linker’, or ‘archiver’.
Returns:: Registry of tools for this language.
Return type:: dict

classmethod get_buildfile_lock(fname=None, context=None, instance=None, **kwargs)[source]¶

Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

Parameters:

name (str) – Build file.
context (threading.Context) – Threading context.
instance (ModelDriver) – Driver instance that should be used.
**kwargs – Additional keyword arguments are passed to the FileLock initialization.

Returns:

Lock for the buildfile.

Return type:

FileLock

classmethod get_compiler_flags(toolname=None, compiler=None, **kwargs)[source]¶

Determine the flags required by the current compiler.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
compiler (CompilerBase, optional) – Compilation tool class for the compiler that should be used. Defaults to None and is set based on toolname.
**kwargs – Keyword arguments are passed to cls.update_compiler_kwargs first and then the compiler’s get_flags method.

Returns:

Flags for the compiler.

Return type:

classmethod get_dependency_include_dirs(dep, toolname=None, default=None)[source]¶

Get the include directories for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns:

Full paths to the directories containing the dependency’s: header(s).

Return type:

Raises:

ValueError – If the include directory cannot be determined for the specified dependency and default is None.

classmethod get_dependency_info(dep, toolname=None, default=None)[source]¶

Get the dictionary of information associated with a dependency.

Parameters:

dep (str) – Name of internal or external dependency.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (dict, optional) – Information dictionary that should be returned if dep cannot be located. Defaults to None and an error will be raised if dep cannot be found.

Returns:

Dependency info.

Return type:

classmethod get_dependency_library(dep, default=None, libtype=None, commtype=None, toolname=None, disable_python_c_api=False, with_asan=False)[source]¶

Get the library location for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.
libtype (str, optional) – Library type that should be returned. Valid values are ‘static’ and ‘shared’. Defaults to None and will be set on the dependency’s libtype (if it has one) or the _default_libtype parameter (if it doesn’t).
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Full path to the library file. For header only libraries,: an empty string will be returned.

Return type:

Raises:

ValueError – If libtype is not ‘static’ or ‘shared’.
ValueError – If the path to the library cannot be determined for the specified dependency and default is None.

classmethod get_dependency_object(dep, default=None, commtype=None, toolname=None, disable_python_c_api=False, with_asan=False)[source]¶

Get the location of an object file for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the object file.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Full path to the object file.

Return type:

classmethod get_dependency_order(deps, toolname=None, disable_python_c_api=False)[source]¶

Get the correct dependency order, including any dependencies for the direct dependencies.

Parameters:

deps (list) – Dependencies in order.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.

Returns:

Dependency order.

Return type:

classmethod get_dependency_source(dep, toolname=None, default=None)[source]¶

Get the path to the library source files (or header files) for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns:

Full path to the library source file. For header only libraries: this will be the header location.

Return type:

classmethod get_external_libraries(no_comm_libs=False)[source]¶

Determine the external libraries that are required based on the default comm.

Parameters:

no_comm_libs (bool, optional) – If True, libraries for the installed comms are not included in the returned list. Defaults to False.

Returns:

The names of external libraries required by the interface: library, including the dependency libraries for the installed comm libraries.

Return type:

classmethod get_internal_suffix(commtype=None, disable_python_c_api=False, with_asan=False)[source]¶

Determine the suffix that should be used for internal libraries.

Parameters:

commtype (str, optional) – If provided, this is the communication type that should be used for the model. If None, the default comm is used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Suffix that should be added to internal libraries to: differentiate between different dependencies.

Return type:

classmethod get_linker_flags(toolname=None, **kwargs)[source]¶

Determine the flags required by the current linker.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Keyword arguments are passed to cls.update_linker_kwargs first and then the linker’s get_flags method.

Returns:

Flags for the linker.

Return type:

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

classmethod get_tool(*args, **kwargs)[source]¶

Get tool from a driver class.

Parameters:

*args – Arguments are passed to the get_tool_static method.
**kwargs – Keyword arguments are passed to the get_tool_static method.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

CompilationToolBase

get_tool_instance(*args, **kwargs)[source]¶

Get tool from a driver instance.

Parameters:

*args – Arguments are passed to the get_tool_static method.
**kwargs – Keyword arguments are passed to the get_tool_static method.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

CompilationToolBase

static get_tool_static(cls, tooltype, toolname=None, return_prop='tool', default=False, language=None)[source]¶

Get the class associated with the specified compilation tool for this language.

Parameters:

cls (class, instance) – Compiled driver class or instance of compiled driver class to get tool for.
tooltype (str) – Type of compilation tool that should be returned.
toolname (str, optional) – Name of the tool that should be returned. Defaults to None and the tool name associated with the provided class/instance will be used.
return_prop (str, optional) – Value that should be returned. If ‘tool’, the tool is returned. If ‘name’, the tool name is returned. If ‘flags’, the tool flags are returned. Defaults to ‘tool’.
default (object, optiona) – Tool that should be returned if one cannot be identified. If False, an error will be raised when a tool cannot be located. Defaults to False.
language (str, optional) – Language of tools that should be returned. Defaults to None if not provided.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

CompilationToolBase

Raises:

NotImplementedError – If a tool is not specified.
ValueError – If return_prop is not ‘tool’, ‘name’, or ‘flags’.

classmethod identify_source_files(args=None, working_dir=None, source_files=None, **kwargs)[source]¶

Determine the source file based on model arguments.

Parameters:

args (list, optional) – Arguments provided.
working_dir (str, optional) – Working directory.
source_files (list, optional) – Source files in the model.
**kwargs – Additional keyword arguments are ignored.

Returns:

Source files.

Return type:

classmethod is_language_installed()[source]¶

Determine if the interpreter/compiler for the associated programming language is installed.

Returns:: True if the language interpreter/compiler is installed.
Return type:: bool

classmethod is_library_installed(lib, cfg=None)[source]¶

Determine if a dependency is installed by check for the appropriate config options setting the path to the library files.

Parameters:

lib (str) – Name of the library that should be checked.
cfg (CisConfigParser, optional) – Config class that should be checked. Defaults to yggdrasil.config.ygg_cfg if not provided.

Returns:

True if the library is installed, False otherwise.

Return type:

classmethod is_standard_library(dep)[source]¶

Determine if a dependency is a standard library.

Parameters:

dep (str, tuple) – Dependency name or tuple with language and dependency name.

Returns:

True if the dependency is a standard library, false: otherwise.

Return type:

classmethod is_tool_installed(tooltype)[source]¶

Determine if a compilation tool of a certain is installed for this language.

Parameters:: tooltype (str) – Type of tool to check for. Supported values include ‘compiler’, ‘linker’, & ‘archiver’.
Returns:: True if a tool of the specified type is installed.
Return type:: bool

classmethod language_executable(toolname=None)[source]¶

Command required to compile/run a model written in this language from the command line.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns:

Name of (or path to) compiler/interpreter executable required: to run the compiler/interpreter from the command line.

Return type:

classmethod language_version(toolname=None, **kwargs)[source]¶

Determine the version of this language.

Parameters:

**kwargs – Keyword arguments are passed to cls.run_executable.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns:

Version of compiler/interpreter for this language.

Return type:

model_command()[source]¶

Return the command that should be used to run the model.

Returns:

Any commands/arguments needed to run the model from the: command line.

Return type:

classmethod mpi_partner_cleanup(self)[source]¶: Actions cleaning up an MPIPartnerModel.

classmethod mpi_partner_init(self)[source]¶: Actions initializing an MPIPartnerModel.

parse_arguments(args, **kwargs)[source]¶

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod partner_buildfile_lock(self)[source]¶: Actions completing buildfile lock on MPIPartnerModels.

run_model(**kwargs)[source]¶

Run the model. Unless overridden, the model will be run using run_executable.

Parameters:: **kwargs – Keyword arguments are passed to run_executable.

classmethod select_suffix_kwargs(kwargs)[source]¶

Select suffix kwargs.

Parameters:: kwargs (dict) – Keyword arguments to select suffix kwargs from.
Returns:: Selected suffix kwargs.
Return type:: dict

set_env(for_compile=False, compile_kwargs=None, toolname=None, **kwargs)[source]¶

Get environment variables that should be set for the model process.

Parameters:

for_compile (bool, optional) – If True, environment variables are set that are necessary for compiling. Defaults to False.
compile_kwargs (dict, optional) – Keyword arguments that should be passed to the compiler’s set_env method. Defaults to empty dict.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

classmethod set_env_compiler(compiler=None, toolname=None, **kwargs)[source]¶

Get environment variables that should be set for the compilation process.

Parameters:

compiler (CompilerBase, optional) – Compiler that set_env shoudl be called for. If not provided, the default compiler for this language will be used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

set_target_language()[source]¶

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns:: Name of language.
Return type:: str

classmethod update_compiler_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, directory=None, include_dirs=None, definitions=None, skip_interface_flags=False, **kwargs)[source]¶

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters:

dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.
for_api (bool, optional) – If True, flags are added that are required for compiling internal api libraries in this language. This includes external communication libraries. Defaults to False.
for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
include_dirs (list, optional) – If provided, each list element will be added as an included directory flag. Defaults to None and is initialized as an empty list.
definitions (list, optional) – If provided, each list element will be added as a defintion. Defaults to None and is initialized to an empty list.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.
internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.
external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.
**kwargs – Additional keyword arguments are passed to the compiler class’s ‘get_flags’ method and get_linker_flags if dont_link is False.

Returns:

Keyword arguments for a get_flags method providing compiler: flags.

Return type:

classmethod update_linker_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, libtype='object', skip_interface_flags=False, use_library_path_internal=False, **kwargs)[source]¶

Update keyword arguments supplied to the linker/archiver get_flags method for various options.

Parameters:

for_api (bool, optional) – If True, flags are added that are required for linking internal api libraries in this language. This includes external communication libraries. Defaults to False.
for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
libtype (str, optional) – Library type that should be created by the linker/archiver. Valid values are ‘static’, ‘shared’, or ‘object’. Defaults to ‘object’.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.
libraries (list, optional) – Full paths to libraries that should be linked against. Defaults to an empty list.
internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.
external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.
use_library_path_internal (bool, optional) – If True, internal dependencies are included as full paths. Defaults to False.
**kwargs – Additional keyword arguments are passed to the linker (or archiver if static is True) ‘get_flags’ method.

Returns:

Keyword arguments for a get_flags method providing linker/: archiver flags.

Return type: