yggdrasil.drivers package

Submodules

yggdrasil.drivers.BuildModelDriver module

class yggdrasil.drivers.BuildModelDriver.BuildModelDriver(*args, **kwargs)[source]

Bases: CompiledModelDriver

Class for running build file compiled drivers.

Parameters
  • name (str) – Driver name.

  • args (str, list) – Executable that should be created (make target) and any arguments for the executable.

  • target (str, optional) – Make target that should be built to create the model executable. Defaults to None.

  • target_language (str, optional) – Language that the target is written in. Defaults to None and will be set based on the source files provided.

  • target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.

  • target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.

  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • env_compiler (str, optional) – Environment variable where the compiler executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.

  • env_compiler_flags (str, optional) – Environment variable where the compiler flags should be stored (including those required to compile against the yggdrasil interface). If not provided, this will be determined by the target language driver.

  • env_linker (str, optional) – Environment variable where the linker executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.

  • env_linker_flags (str, optional) – Environment variable where the linker flags should be stored (including those required to link against the yggdrasil interface). If not provided, this will be determined by the target language driver.

  • **kwargs – Additional keyword arguments are passed to parent class.

buildfile

Path to file containing build instructions.

Type

str

builddir

Path to directory where build products will be saved.

Type

str

sourcedir

Path to directory where source files are located.

Type

str

compiledir

Path to directory where compilation call should be made from. Defaults to working_dir.

Type

str

target

Name of executable that should be created and called.

Type

str

target_language

Language that the target is written in.

Type

str

target_language_driver

Language driver for the target language.

Type

ModelDriver

target_compiler

Compilation tool that should be used to compile the target language.

Type

str

target_linker

Compilation tool that should be used to link the target language.

Type

str

target_compiler_flags

Compilation flags that should be passed to the target language compiler.

Type

list

target_linker_flags

Linking flags that should be passed to the target language linker.

Type

list

env_compiler

Compiler environment variable.

Type

str

env_compiler_flags

Compiler flags environment variable.

Type

str

env_linker

Linker environment variable.

Type

str

env_linker_flags

Linker flags environment variable.

Type

str

Class Attributes:
built_where_called (bool): If True, it is assumed that compilation

output will be saved in the same directory from which the compilation command is issued.

Raises

RuntimeError – If neither the IPC or ZMQ C libraries are available.

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

allow_parallel_build = False
base_languages = ['c']
buildfile_base = None
built_where_called = False
classmethod call_compiler(src, **kwargs)[source]

Compile a source file into an executable or linkable object file, checking for errors.

Parameters
  • src (str) – Full path to source file.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Full path to compiled source.

Return type

str

cleanup()[source]

Remove compiled executable.

compile_dependencies_instance(*args, **kwargs)[source]

Compile dependencies specifically for this instance.

compile_model(**kwargs)[source]

Compile model executable(s).

Parameters

**kwargs – Keyword arguments are passed on to the parent class’s method.

executable_type = 'build'
classmethod fix_path(path, for_env=False)[source]

Update a path.

Parameters
  • path (str) – Path that should be formatted.

  • for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.

Returns

Updated path.

Return type

str

full_language = False
classmethod get_buildfile_lock(**kwargs)[source]

Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

classmethod get_language_for_buildfile(buildfile, target=None)[source]

Determine the target language based on the contents of a build file.

Parameters
  • buildfile (str) – Full path to the build configuration file.

  • target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_language_for_source(fname=None, buildfile=None, languages=None, early_exit=False, call_base=False, **kwargs)[source]

Determine the language that can be used with the provided source file(s). If more than one language applies to a set of multiple files, the language that applies to the most files is returned.

Parameters
  • fname (str, list) – The full path to one or more files. If more than one is provided, they are iterated over.

  • buildfile (str, optional) – Full path to the build configuration file. Defaults to None and will be searched for.

  • languages (list, optional) – The list of languages that are acceptable. Defaults to None and any language will be acceptable.

  • early_exit (bool, optional) – If True, the first language identified will be returned if fname is a list of files. Defaults to False.

  • source_dir (str, optional) – Full path to the directory containing the source files. Defaults to None and is determiend from fname.

  • buildfile – Full path to the build configuration file. Defaults to None and will be searched for.

  • target (str, optional) – The build target. Defaults to None.

  • call_base (bool, optional) – If True, the base class’s method is called directly. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The language that can operate on the specified file.

Return type

str

classmethod get_source_dir(fname=None, source_dir=None)[source]
classmethod get_target_language_info(target_language_driver=None, target_language='c', target_compiler=None, target_compiler_flags=None, target_linker=None, target_linker_flags=None, logging_level=None, without_wrapper=False, compiler_flag_kwargs=None, linker_flag_kwargs=None, **kwargs)[source]

Get a dictionary of information about language compilation tools.

Parameters
  • target_language_driver (ModelDriver, optional) – Driver associated with the target language. If not provided, one will be created based on ‘target_language’.

  • target_language (str, optional) – Language to get info for. Defaults to ‘c’.

  • target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.

  • target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.

  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • logging_level (int, optional) – The numeric logging level that should be passed as a definition. Defaults to None and is ignored.

  • without_wrapper (bool, optional) – If True, the returned info will be updated for compilation when a build wrapper was not created. Defaults to False.

  • compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.

  • linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.

  • **kwargs – Additional keyword arguments are added to the output dictionary.

Returns

Information about language compilers and linkers.

Return type

dict

is_build_tool = True
classmethod is_source_file(fname)[source]

Determine if the provided file name points to a source files for the associated programming language by checking the extension.

Parameters

fname (str) – Path to file.

Returns

True if the provided file is a source file, False otherwise.

Return type

bool

parse_arguments(args, **kwargs)[source]

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters
  • args (list) – List of arguments provided.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

set_env(**kwargs)[source]

Get environment variables that should be set for the model process.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

classmethod set_env_compiler(language_info=None, **kwargs)[source]

Get environment variables that should be set for the compilation process.

Parameters
  • language_info (dict) – Language compilation tool information.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

set_target_language()[source]

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns

Name of language.

Return type

str

sourcedir_as_sourcefile = False
property target_language_info

Information about the underlying language.

Type

dict

class yggdrasil.drivers.BuildModelDriver.BuildToolBase(**kwargs)[source]

Bases: CompilerBase

Base class for build tools.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = None
classmethod get_flags(**kwargs)[source]

Get compilation flags, replacing outfile with target.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

classmethod get_tool_suffix()[source]

Get the string that should be added to tool products based on the tool used.

Returns

Suffix that should be added to tool products to indicate the

tool used.

Return type

str

is_build_tool = True

yggdrasil.drivers.CMakeModelDriver module

class yggdrasil.drivers.CMakeModelDriver.CMakeBuilder(**kwargs)[source]

Bases: LinkerBase

CMake build tool.

classmethod call(*args, **kwargs)[source]

Print contents of CMakeCache.txt before raising error.

default_executable = 'cmake'
default_flags = []
executable_ext = ''
classmethod extract_kwargs(kwargs, **kwargs_ex)[source]

Extract linker kwargs, leaving behind just compiler kwargs.

Parameters
  • kwargs (dict) – Keyword arguments passed to the compiler that should be sorted into kwargs used by either the compiler or linker or both. Keywords that are not used by the compiler will be removed from this dictionary.

  • **kwargs_ex – Additional keyword arguments are passed to the parent class’s method.

Returns

Keyword arguments that should be passed to the linker.

Return type

dict

flag_options = {'builddir': {'key': '--build', 'position': 0}, 'configuration': '--config', 'target': '--target'}
classmethod get_executable_command(args, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(target=None, builddir=None, **kwargs)[source]

Get a list of build flags for building a project using cmake.

Parameters
  • target (str, optional) – Target that should be built. Defaults to to None and is ignored.

  • builddir (str, optional) – Directory containing the build tree. Defaults to None and is set based on outfile is provided or cls.default_builddir if not. Defaults to ‘.’ (which will be the current working directory).

  • **kwargs – Additional keyword arguments are ignored.

Returns

Linker flags.

Return type

list

classmethod get_output_file(obj, target=None, builddir=None, **kwargs)[source]

Determine the appropriate output file that will result when bulding a given directory.

Parameters
  • obj (str) – Directory being built or a file in the directory being built.

  • target (str, optional) – Target that will be used to create the output file. Defaults to None. Target is required in order to determine the name of the file that will be created.

  • builddir (str, optional) – Directory where build tree should be created. Defaults to None and obj will used (if its a directory) or the directory containing obj will be used (if its a file).

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Full path to file that will be produced.

Return type

str

Raises

RuntimeError – If target is None.

is_gnu = False
languages = ['cmake']
library_ext = '.so'
output_key = None
tool_suffix_format = ''
toolname = 'cmake'
class yggdrasil.drivers.CMakeModelDriver.CMakeConfigure(**kwargs)[source]

Bases: BuildToolBase

CMake configuration tool.

add_libraries = False
classmethod append_product(products, src, new, **kwargs)[source]

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters
  • products (list) – List of of existing products that new product should be appended to.

  • src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).

  • new (str) – New product that should be appended to the list.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = 'cmake'
classmethod call(args, **kwargs)[source]

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters
  • args (list) – The arguments that should be passed to the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method and the associated linker/archiver’s call method if dont_link is False.

Returns

Output to stdout from the command execution if skip_flags is

True, produced file otherwise.

Return type

str

combine_with_linker = False
compile_only_flag = None
classmethod create_include(fname, target, compiler=None, compiler_flags=None, linker=None, linker_flags=None, library_flags=None, internal_library_flags=None, configuration='Release', verbose=False, **kwargs)[source]

Create CMakeList include file with necessary includes, definitions, and linker flags.

Parameters
  • fname (str) – File where the include file should be saved.

  • target (str) – Target that links should be added to.

  • compiler (CompilerBase) – Compiler that should be used to generate the list of compilation flags.

  • compile_flags (list, optional) – Additional compile flags that should be set. Defaults to [].

  • linker (LinkerBase) – Linker that should be used to generate the list of compilation flags.

  • linker_flags (list, optional) – Additional linker flags that should be set. Defaults to [].

  • library_flags (list, optional) – List of library flags to add. Defaults to [].

  • internal_library_flags (list, optional) – List of library flags associated with yggdrasil libraries. Defaults to [].

  • configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’. Only used on Windows to determin the standard library.

  • verbose (bool, optional) – If True, the contents of the created file are displayed. Defaults to False.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Lines that should be added before the executable is defined

in the CMakeLists.txt (e.g. LINK_DIRECTORIES commands).

Return type

list

Raises

ValueError – If a linker or compiler flag cannot be interpreted.

default_archiver = False
default_builddir = '.'
default_executable = 'cmake'
default_flags = []
classmethod fix_path(x, is_gnu=False)[source]

Fix paths so that they conform to the format expected by the OS and/or build tool.

flag_options = {'builddir': '-B%s', 'configuration': '-DCMAKE_BUILD_TYPE=%s', 'definitions': '-D%s', 'generator': '-G%s', 'platform': '-A%s', 'sourcedir': '', 'toolset': '-T%s'}
classmethod generator(return_default=False, default=None, **kwargs)[source]

Determine the generator that should be used.

Parameters
  • return_default (bool, optional) – If True, the default generator will be returned even if the environment variable is set. Defaults to False.

  • default (str, optional) – Value that should be returned if a generator cannot be located. Defaults to None.

  • **kwargs – Keyword arguments are passed to cls.call.

Returns

Name of the generator.

Return type

str

classmethod generator2toolset(generator)[source]

Determine the toolset string option that corresponds to the provided generator name.

Parameters

generator (str) – Name of the generator.

Returns

Name of the toolset.

Return type

str

Raises
  • NotImplementedError – If the platform is not windows.

  • ValueError – If the generator is not a flavor of Visual Studio.

  • ValueError – If a tool set cannot be located for the specified generator.

classmethod get_executable_command(args, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(sourcedir='.', builddir=None, target_compiler=None, target_linker=None, **kwargs)[source]

Get a list of configuration/generation flags.

Parameters
  • sourcedir (str, optional) – Directory containing the source files to be compiled and the target CMakeLists.txt file. Defaults to ‘.’ (the current working directory).

  • builddir (str, optional) – Directory that will contain the build tree. Defaults to ‘.’ (this current working directory).

  • target_compiler (str, optional) – Compiler that should be used by cmake. Defaults to None and the default for the target language will be used.

  • target_linker (str, optional) – Linker that should be used by cmake. Defaults to None and the default for the target language will be used.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

Raises
  • RuntimeError – If dont_link is True and the provide outfile and builddir keyword arguments point to conflicting paths.

  • ValueError – If ‘include_dirs’ is set (‘sourcedir’ should be used for cmake to specify the location of the source).

classmethod get_output_file(src, dont_link=False, dont_build=None, sourcedir=None, builddir=None, working_dir=None, **kwargs)[source]

Determine the appropriate output file or directory that will result when configuring/building a given source directory.

Parameters
  • src (str) – Directory containing source files being compiled.

  • dont_link (bool, optional) – If True, the result assumes that the source is just compiled and not linked. If False, the result will be the final result after linking.

  • dont_build (bool, optional) – Alias for dont_link. If not None, this keyword overrides the value of dont_link. Defaults to None.

  • sourcedir (str, optional) – Directory where sources files are located. Defaults to None. If None, src will be used to determine the value.

  • builddir (str, optional) – Directory where build tree should be created. Defaults to None. If None, sourcedir will be used.

  • working_dir (str, optional) – Working directory where output file should be located. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are ignored unless dont_link is False; then they are passed to get_linker_output_file

Returns

Full path to file that will be produced.

Return type

str

is_gnu = False
is_linker = False
languages = ['cmake']
output_key = None
product_files = ['Makefile', 'CMakeCache.txt', 'cmake_install.cmake', 'CMakeFiles']
remove_product_exts = ['CMakeFiles']
toolname = 'cmake'
class yggdrasil.drivers.CMakeModelDriver.CMakeModelDriver(*args, **kwargs)[source]

Bases: BuildModelDriver

Class for running cmake compiled drivers. Before running the cmake command, the cmake commands for setting the necessary compiler & linker flags for the interface’s C/C++ library are written to a file called ‘ygg_cmake.txt’ that should be included in the CMakeLists.txt file (after the target executable has been added).

Parameters
  • name (str) – Driver name.

  • args (str, list) – Executable that should be created (cmake target) and any arguments for the executable.

  • sourcedir (str, optional) – Source directory to call cmake on. If not provided it is set to working_dir. This should be the directory containing the CMakeLists.txt file. It can be relative to working_dir or absolute.

  • builddir (str, optional) – Directory where the build should be saved. Defaults to <sourcedir>/build. It can be relative to working_dir or absolute.

  • configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’.

  • **kwargs – Additional keyword arguments are passed to parent class.

sourcedir

Source directory to call cmake on.

Type

str

add_libraries

If True, interface libraries and dependency libraries are added using CMake’s ADD_LIBRARY directive. If False, interface libraries are found using FIND_LIBRARY.

Type

bool

configuration

Build type/configuration that should be built. This is only used on Windows.

Type

str

Raises

RuntimeError – If neither the IPC or ZMQ C libraries are available.

add_libraries = False
buildfile_base = 'CMakeLists.txt'
property buildfile_orig

Full path to where the original CMakeLists.txt file will be stored during compilation of the modified file.

Type

str

property buildfile_ygg

Full path to the verison of the CMakeLists.txt that has been updated w/ yggdrasil compilation flags.

Type

str

cfg = <yggdrasil.config.YggConfigParser object>
compile_model(target=None, **kwargs)[source]

Compile model executable(s) and appends any products produced by the compilation that should be removed after the run is complete.

Parameters
  • target (str, optional) – Target to build.

  • **kwargs – Keyword arguments are passed on to the call_compiler method.

default_archiver = None
default_archiver_flags = None
default_compiler = 'cmake'
default_compiler_flags = None
default_linker = 'cmake'
default_linker_flags = None
classmethod fix_path(path, for_env=False, **kwargs)[source]

Update a path.

Parameters
  • path (str) – Path that should be formatted.

  • for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Updated path.

Return type

str

classmethod get_language_for_buildfile(buildfile, target=None)[source]

Determine the target language based on the contents of a build file.

Parameters
  • buildfile (str) – Full path to the build configuration file.

  • target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_target_language_info(target_compiler_flags=None, target_linker_flags=None, compiler_flag_kwargs=None, linker_flag_kwargs=None, without_wrapper=False, **kwargs)[source]

Get a dictionary of information about language compilation tools.

Parameters
  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.

  • linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Information about language compilers and linkers.

Return type

dict

inverse_type_map = None
language = 'cmake'
parse_arguments(args, **kwargs)[source]

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters
  • args (list) – List of arguments provided.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod prune_sh_gcc(path, gcc)[source]

Remove instances of sh.exe from the path that are not associated with the selected gcc compiler. This can happen on windows when rtools or git install a version of sh.exe that is added to the path before the compiler.

Parameters
  • path (str) – Contents of the path variable.

  • gcc (str) – Full path to the gcc executable.

Returns

Modified path that removes the extra instances

of sh.exe.

Return type

str

sourcedir_as_sourcefile = True
classmethod update_compiler_kwargs(**kwargs)[source]

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Keyword arguments for a get_flags method providing compiler

flags.

Return type

dict

use_env_vars = False
write_wrappers(**kwargs)[source]

Write any wrappers needed to compile and/or run a model.

Parameters

**kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Full paths to any created wrappers.

Return type

list

yggdrasil.drivers.CModelDriver module

class yggdrasil.drivers.CModelDriver.ARArchiver(**kwargs)[source]

Bases: ArchiverBase

Archiver class for ar tool.

compatible_toolsets = ['gnu', 'llvm']
default_executable = 'ar'
default_executable_env = 'AR'
default_flags = []
default_flags_env = None
is_gnu = True
languages = ['c', 'c++', 'fortran']
library_ext = '.a'
output_first_library = True
output_key = ''
search_path_envvar = ['LIBRARY_PATH']
shared_library_flag = None
static_library_flag = 'rcs'
toolname = 'ar'
toolset = 'gnu'
class yggdrasil.drivers.CModelDriver.CCompilerBase(**kwargs)[source]

Bases: CompilerBase

Base class for C compilers.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

classmethod call(args, **kwargs)[source]

Call the compiler with the provided arguments. For yggdrasil C models will always be linked using the C++ linker since some parts of the interface library are written in C++.

default_executable_env = 'CC'
default_flags = ['-g', '-Wall']
default_flags_env = 'CFLAGS'
classmethod get_search_path(*args, **kwargs)[source]

Determine the paths searched by the tool for external library files.

Parameters
  • *args – Additional arguments are passed to the parent class’s method.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

List of paths that the tools will search.

Return type

list

languages = ['c']
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH']}
search_path_envvar = ['C_INCLUDE_PATH']
search_path_flags = ['-E', '-v', '-xc', '/dev/null']
search_regex = ['(?:#include <...> search starts here:)|(?: ([^\\n]+?)(?: \\(framework directory\\))?)\\n']
search_regex_begin = '#include "..." search starts here:'
search_regex_end = 'End of search list.'
classmethod set_env(*args, **kwargs)[source]

Set environment variables required for compilation.

Parameters
  • *args – Arguments are passed to the parent class’s method.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

class yggdrasil.drivers.CModelDriver.CModelDriver(*args, **kwargs)[source]

Bases: CompiledModelDriver

Class for running C models.

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration.

classmethod allows_realloc(var)[source]

Determine if a variable allows the receive call to perform realloc.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if the variable allows realloc, False otherwise.

Return type

bool

brackets = ('{', '}')
classmethod call_linker(obj, language=None, **kwargs)[source]

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters
  • obj (list) – Object files that should be linked.

  • language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.

  • **kwargs – Additional keyword arguments are passed to run_executable.

Returns

Full path to compiled source.

Return type

str

cfg = <yggdrasil.config.YggConfigParser object>
classmethod configure(cfg, macos_sdkroot=None, vcpkg_dir=None, **kwargs)[source]

Add configuration options for this language. This includes locating any required external libraries and setting option defaults.

Parameters
  • cfg (YggConfigParser) – Config class that options should be set for.

  • macos_sdkroot (str, optional) – Full path to the root directory for the MacOS SDK that should be used. Defaults to None and is ignored.

  • vcpkg_dir (str, optional) – Full path to the root directory containing a vcpkg installation. This should be the directory that contains the vcpkg executable and any packages installed by vcpkg (in subdirectories). Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

default_archiver = 'ar'
default_archiver_flags = None
default_compiler = 'gcc'
default_compiler_flags = None
default_linker = 'gcc'
default_linker_flags = None
external_libraries = {'czmq': {'include': 'czmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libczmq.so'}, 'numpy': {'include': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h', 'language': 'c', 'libtype': 'header_only'}, 'python': {'include': '/opt/hostedtoolcache/Python/3.8.13/x64/include/python3.8/Python.h', 'language': 'c', 'libtype': 'shared', 'shared': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/libpython3.8.so'}, 'rapidjson': {'include': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/rapidjson/include/rapidjson/rapidjson.h', 'language': 'c', 'libtype': 'header_only'}, 'zmq': {'include': 'zmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libzmq.so'}}
classmethod finalize_function_io(direction, x)[source]

Finalize info for an input/output channel following function parsing.

Parameters
  • direction (str) – Direction of channel (‘input’ or ‘output’)

  • x (dict) – Channel info.

classmethod format_function_param(key, default=None, **kwargs)[source]

Return the formatted version of the specified key.

Parameters
  • key (str) – Key in cls.function_param mapping that should be formatted.

  • default (str, optional) – Format that should be returned if key is not in cls.function_param. Defaults to None.

  • **kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns

Formatted string.

Return type

str

Raises

NotImplementedError – If key is not in cls.function_param and default is not set.

function_param = {'and': '&&', 'assign': '{name} = {value};', 'assign_copy': 'memcpy({name}, {value}, {N}*sizeof({native_type}));', 'block_end': '}', 'break': 'break;', 'comment': '//', 'copy_any': '{name} = copy_generic({value});', 'copy_array': '{name} = copy_json_array({value});', 'copy_class': '{name} = copy_python({value});', 'copy_function': '{name} = copy_python({value});', 'copy_instance': '{name} = copy_generic({value});', 'copy_obj': '{name} = copy_obj({value});', 'copy_object': '{name} = copy_json_object({value});', 'copy_ply': '{name} = copy_ply({value});', 'copy_schema': '{name} = copy_schema({value});', 'declare': '{type_name} {variable};', 'error': 'printf("{error_msg}\\n"); return -1;', 'exec_begin': 'int main() {', 'exec_end': '  return 0;\n}', 'exec_prefix': '#include <stdbool.h>', 'false': 'false', 'false_flag': '0', 'flag_cond': '{flag_var} >= 0', 'for_begin': 'for ({iter_var} = {iter_begin}; {iter_var} < {iter_end}; {iter_var}++) {{', 'fprintf': 'printf("{message}\\n", {variables});', 'free': 'if ({variable} != NULL) {{ free({variable}); {variable} = NULL; }}', 'free_any': 'free_generic({variable});', 'free_array': 'free_json_array({variable});', 'free_class': 'destroy_python({variable});', 'free_function': 'destroy_python({variable});', 'free_instance': 'free_generic({variable});', 'free_obj': 'free_obj({variable});', 'free_object': 'free_json_object({variable});', 'free_ply': 'free_ply({variable});', 'free_schema': 'free_schema({variable});', 'function_def_begin': '{output_type} {function_name}({input_var}) {{', 'function_def_regex': '(?P<flag_type>.+?)\\s*{function_name}\\s*\\((?P<inputs>(?:[^{{])*?)\\)\\s*\\{{(?P<body>(?:.*?\\n?)*?)(?:(?:return *(?P<flag_var>.+?)?;(?:.*?\\n?)*?\\}})|(?:\\}}))', 'if_begin': 'if ({cond}) {{', 'if_elif': '}} else if ({cond}) {{', 'if_else': '}} else {{', 'import': '#include "{filename}"', 'indent': '  ', 'index': '{variable}[{index}]', 'init_any': 'init_generic()', 'init_array': 'init_json_array()', 'init_class': 'init_python()', 'init_function': 'init_python()', 'init_instance': 'init_generic()', 'init_obj': 'init_obj()', 'init_object': 'init_json_object()', 'init_ply': 'init_ply()', 'init_schema': 'init_schema()', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray_arr("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': '{channel} = yggInputType("{channel_name}", {channel_type});', 'inputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(\\((?P<name_ptr>\\*+)?)?(?P<name>.+?)(?(4)(?:\\)))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'interface': '#include "{interface_library}"', 'line_end': ';', 'not': '!', 'not_flag_cond': '{flag_var} < 0', 'null': 'NULL', 'output': '{channel} = yggOutputType("{channel_name}", {channel_type});', 'outputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(?P<name>.+?)(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'print': 'printf("{message}\\n");', 'print_any': 'display_generic({object});', 'print_array': 'display_json_array({object});', 'print_bytes': 'printf("%s\\n", {object});', 'print_class': 'display_python({object});', 'print_complex': 'print_complex({object});', 'print_float': 'printf("%f\\n", {object});', 'print_function': 'display_python({object});', 'print_instance': 'display_generic({object});', 'print_int': 'printf("%i\\n", {object});', 'print_obj': 'display_obj({object});', 'print_object': 'display_json_object({object});', 'print_ply': 'display_ply({object});', 'print_schema': 'display_schema({object});', 'print_string': 'printf("%s\\n", {object});', 'print_uint': 'printf("%u\\n", {object});', 'print_unicode': 'printf("%s\\n", {object});', 'quote': '"', 'recv_function': 'yggRecvRealloc', 'recv_heap': 'yggRecvRealloc', 'recv_stack': 'yggRecv', 'return': 'return {output_var};', 'send_function': 'yggSend', 'true': 'true', 'true_flag': '1', 'while_begin': 'while ({cond}) {{'}
classmethod get_dependency_info(dep, toolname=None, default=None)[source]

Get the dictionary of information associated with a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the library.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • default (dict, optional) – Information dictionary that should be returned if dep cannot be located. Defaults to None and an error will be raised if dep cannot be found.

Returns

Dependency info.

Return type

dict

classmethod get_json_type(native_type)[source]

Get the JSON type from the native language type.

Parameters

native_type (str) – The native language type.

Returns

The JSON type.

Return type

str, dict

classmethod get_name_declare(var)[source]

Determine the name that should be used for declaration.

Parameters

var (str, dict) – Name of variable or dictionary of information.

Returns

Modified name for declaration.

Return type

str

classmethod get_native_type(**kwargs)[source]

Get the native type.

Parameters
  • type (str, optional) – Name of yggdrasil extended JSON type or JSONSchema dictionary defining a datatype.

  • **kwargs – Additional keyword arguments may be used in determining the precise declaration that should be used.

Returns

The native type.

Return type

str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class.

Returns

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type

dict

include_channel_obj = True
classmethod input2output(var)[source]

Perform conversion necessary to turn a variable extracted from a function definition from an input to an output.

Parameters

var (dict) – Variable definition.

Returns

Updated variable definition.

Return type

dict

interface_dependencies = ['rapidjson']
interface_directories = ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C']
interface_library = 'ygg'
interface_map = {'call': 'flag = rpcCall({channel_obj}, {outputs}, {inputs})', 'client': 'yggRpcClientType("{channel_name}", {datatype_out}, {datatype_in})', 'import': '#include "YggInterface.h"', 'input': 'yggInputType("{channel_name}", {datatype})', 'output': 'yggOutputType("{channel_name}", {datatype})', 'recv': 'flag = yggRecv({channel_obj}, {input_refs})', 'send': 'flag = yggSend({channel_obj}, {outputs})', 'server': 'yggRpcServerType("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'yggTimesync("{channel_name}", "{time_units}")'}
internal_libraries = {'datatypes': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/datatypes', 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': [], 'internal_dependencies': ['regex'], 'language': 'c++', 'libtype': 'object'}, 'python_wrapper': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C', 'external_dependencies': ['python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'language': 'c', 'libtype': 'shared', 'linker_language': 'c', 'source': 'python_wrapper.c'}, 'regex': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_posix': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_win32': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c++', 'libtype': 'object', 'source': 'regex_win32.cpp'}, 'ygg': {'compiler_flags': ['-fPIC'], 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'internal_dependencies': ['regex', 'datatypes', 'python_wrapper'], 'language': 'c', 'linker_language': 'c++', 'source': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/YggInterface.c'}}
inverse_type_map = None
is_typed = True
language = 'c'
language_ext = ['.c', '.h']
locked_buildfile = 'datatypes.o'
classmethod output2input(var, in_definition=True)[source]

Perform conversion necessary to turn an output variable into an corresponding input that can be used to format a function definition.

Parameters
  • var (dict) – Variable definition.

  • in_definition (bool, optional) – If True, the returned dictionary corresponds to an input variable in a function definition. If False, the returned value will correspond to an input to a function. Defaults to True.

Returns

Updated variable definition.

Return type

dict

outputs_in_inputs = True
classmethod parse_var_definition(io, value, **kwargs)[source]

Extract information about input/output variables from a string definition.

Parameters
  • io (str) – Description of variables contained in the provided string. Must be ‘inputs’ or ‘outputs’.

  • value (str) – String containing one or more variable definitions.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

List of information about the variables contained in

the provided string.

Return type

list

Raises
classmethod prepare_output_variables(vars_list, in_definition=False, in_inputs=False, for_yggdrasil=False)[source]

Concatenate a set of output variables such that it can be passed as a single string to the function_call parameter.

Parameters
  • vars_list (list) – List of variable names to concatenate as output from a function call.

  • in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying output variables in a function definition. Defaults to False.

  • in_inputs (bool, optional) – If True, the output variables should be formated to be included as input variables. Defaults to False.

  • for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns

Concatentated variables list.

Return type

str

classmethod prepare_variables(vars_list, in_definition=False, for_yggdrasil=False)[source]

Concatenate a set of input variables such that it can be passed as a single string to the function_call parameter.

Parameters
  • vars_list (list) – List of variable dictionaries containing info (e.g. names) that should be used to prepare a string representing input/output to/from a function call.

  • in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying variables in a function definition. Defaults to False.

  • for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns

Concatentated variables list.

Return type

str

classmethod requires_length_var(var)[source]

Determine if a variable requires a separate length variable.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if a length variable is required, False otherwise.

Return type

bool

classmethod requires_shape_var(var)[source]

Determine if a variable requires a separate shape variable.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if a shape variable is required, False otherwise.

Return type

bool

classmethod set_env_class(**kwargs)[source]

Set environment variables that are instance independent.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.

Returns

Environment variables for the model process.

Return type

dict

supported_comm_options = {'ipc': {'platforms': ['MacOS', 'Linux']}, 'zmq': {'libraries': ['zmq', 'czmq']}}
supported_comms = ['ipc', 'zmq']
type_class_map = {}
type_map = {'1darray': '*', 'any': 'generic_t', 'array': 'json_array_t', 'boolean': 'bool', 'bytes': 'char*', 'class': 'python_class_t', 'comm': 'comm_t*', 'complex': 'complex_X', 'dtype': 'dtype_t*', 'flag': 'int', 'float': 'double', 'function': 'python_function_t', 'instance': 'python_instance_t', 'int': 'intX_t', 'integer': 'int', 'ndarray': '*', 'null': 'void*', 'obj': 'obj_t', 'object': 'json_object_t', 'ply': 'ply_t', 'schema': 'schema_t', 'string': 'string_t', 'uint': 'uintX_t', 'unicode': 'unicode_t'}
classmethod update_ld_library_path(env, paths_to_add=None, add_to_front=False, add_libpython_dir=False, toolname=None, env_var=None, **kwargs)[source]

Update provided dictionary of environment variables so that LD_LIBRARY_PATH includes the interface directory containing the interface libraries.

Parameters
  • env (dict) – Dictionary of enviroment variables to be updated.

  • paths_to_add (list, optional) – Paths that should be added. If not provided, defaults to [cls.get_language_dir()].

  • add_to_front (bool, optional) – If True, new paths are added to the front, rather than the end. Defaults to False.

  • add_libpython_dir (bool, optional) – If True, the directory containing the Python C library will be added. Defaults to False.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • env_var (str, optional) – Environment variable where the paths should be added. Defaults to None and is only set for linux (LD_LIBRARY_PATH) and windows (PATH).

  • **kwargs – Additional keyword arguments are ignored.

Returns

Updated dictionary of environment variables.

Return type

dict

classmethod update_python_path(env)[source]

Update provided dictionary of environment variables so that PYTHONPATH and PYTHONHOME are set as needed (primarily on windows).

Parameters

env (dict) – Dictionary of enviroment variables to be updated.

Returns

Updated dictionary of environment variables.

Return type

dict

classmethod write_assign_to_output(dst_var, src_var, outputs_in_inputs=False, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]

Write lines assigning a value to an output variable.

Parameters
  • dst_var (str, dict) – Name or information dictionary for variable being assigned to.

  • src_var (str, dict) – Name or information dictionary for value being assigned to dst_var.

  • outputs_in_inputs (bool, optional) – If True, outputs are passed as input parameters. In some languages, this means that a pointer or reference is passed (e.g. C) and so the assignment should be to the memory indicated rather than the variable. Defaults to False.

  • dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.

  • use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines achieving assignment.

Return type

list

classmethod write_declaration(var, **kwargs)[source]

Return the lines required to declare a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Addition keyword arguments are passed to the parent class’s method.

Returns

The lines declaring the variable.

Return type

list

classmethod write_free(var, **kwargs)[source]

Return the lines required to free a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines freeing the variable.

Return type

list

classmethod write_function_def(function_name, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]

Write a function definition.

Parameters
  • function_name (str) – Name fo the function being defined.

  • dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.

  • use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines completing the function call.

Return type

list

Raises

ValueError – If outputs_in_inputs is not True and more than one output variable is specified.

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]

Get the lines necessary to initialize an array for iteration output.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being initialized.

  • value (str, optional) – Value that should be assigned to the variable.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines initializing the variable.

Return type

list

classmethod write_model_function_call(model_function, flag_var, inputs, outputs, **kwargs)[source]

Write lines necessary to call the model function.

Parameters
  • model_function (str) – Handle of the model function that should be called.

  • flag_var (str) – Name of variable that should be used as a flag.

  • inputs (list) – List of dictionaries describing inputs to the model.

  • outputs (list) – List of dictionaries describing outputs from the model.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a call to a model function in

this language.

Return type

list

classmethod write_model_recv(channel, recv_var, **kwargs)[source]

Write a model receive call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being received from was stored in.

  • recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a receive call in this language.

Return type

list

classmethod write_print_output_var(var, in_inputs=False, **kwargs)[source]

Get the lines necessary to print an output variable in this language.

Parameters
  • var (dict) – Variable information.

  • in_inputs (bool, optional) – If True, the output variable is passed in as an input variable to be populated. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to write_print_var.

Returns

Lines printing the specified variable.

Return type

list

class yggdrasil.drivers.CModelDriver.ClangCompiler(**kwargs)[source]

Bases: CCompilerBase

Interface class for clang compiler/linker.

combine_with_linker = True
compatible_toolsets = ['llvm']
default_archiver = 'libtool'
default_executable = 'clang'
default_flags = ['-g', '-Wall']
default_linker = 'clang'
flag_options = {'definitions': '-D%s', 'include_dirs': '-I%s', 'isysroot': {'key': '-isysroot', 'prepend': True}, 'mmacosx-version-min': '-mmacosx-version-min=%s', 'sysroot': '--sysroot'}
classmethod get_flags(*args, **kwargs)[source]

Get a list of compiler flags.

is_gnu = False
is_linker = False
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);']}
platforms = ['MacOS', 'Linux', 'Windows']
toolname = 'clang'
toolset = 'llvm'
class yggdrasil.drivers.CModelDriver.ClangLinker(**kwargs)[source]

Bases: LDLinker

Interface class for clang linker (calls to ld).

aliases = []
static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

compatible_toolsets = ['llvm']
default_executable = 'clang'
default_flags = []
flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': '', 'library_rpath': '-rpath', 'linker-version': '-mlinker-version=%s'}
classmethod get_flags(*args, **kwargs)[source]

Get a list of linker flags.

is_gnu = False
languages = ['c']
library_ext = '.so'
platforms = ['MacOS', 'Linux', 'Windows']
search_path_flags = ['-Xlinker', '-v']
search_regex = ['\\t([^\\t\\n]+)\\n']
search_regex_begin = 'Library search paths:'
toolname = 'clang'
toolset = 'llvm'
class yggdrasil.drivers.CModelDriver.GCCCompiler(**kwargs)[source]

Bases: CCompilerBase

Interface class for gcc compiler/linker.

aliases = ['gnu-cc']
combine_with_linker = True
compatible_toolsets = ['gnu']
default_archiver = 'ar'
default_executable = 'gcc'
default_flags = ['-g', '-Wall']
default_linker = 'gcc'
dll2a(dll, dst=None, overwrite=False)[source]

Convert a window’s .dll library into a static library.

Parameters
  • dll (str) – Full path to .dll library to convert.

  • dst (str, optional) – Full path to location where the new library should be saved. Defaults to None and will be set based on lib or will be placed in the same directory as dll.

  • overwrite (bool, optional) – If True, the static file will be created even if it already exists. Defaults to False.

Returns

Full path to new .a static library.

Return type

str

is_gnu = True
classmethod is_installed()[source]

Determine if this tool is installed by looking for the executable.

Returns

True if the tool is installed, False otherwise.

Return type

bool

linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'flag_options': {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}, 'library_rpath': '-Wl,-rpath'}, 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);']}
platforms = ['MacOS', 'Linux', 'Windows']
toolname = 'gcc'
toolset = 'gnu'
class yggdrasil.drivers.CModelDriver.LDLinker(**kwargs)[source]

Bases: LinkerBase

Linker class for ld tool.

default_executable = 'ld'
default_executable_env = 'LD'
default_flags = []
default_flags_env = 'LDFLAGS'
is_gnu = False
languages = ['c']
library_ext = '.so'
search_path_envvar = ['LIBRARY_PATH', 'LD_LIBRARY_PATH']
classmethod tool_version(**kwargs)[source]

Determine the version of this tool.

Parameters

**kwargs – Keyword arguments are passed to cls.call.

Returns

Version of the tool.

Return type

str

toolname = 'ld'
version_flags = ['-v']
class yggdrasil.drivers.CModelDriver.LibtoolArchiver(**kwargs)[source]

Bases: ArchiverBase

Archiver class for libtool tool.

compatible_toolsets = ['clang']
default_executable = 'libtool'
default_executable_env = 'LIBTOOL'
default_flags = []
is_gnu = False
languages = ['c', 'c++']
library_ext = '.a'
search_path_envvar = ['LIBRARY_PATH']
shared_library_flag = None
static_library_flag = '-static'
toolname = 'libtool'
toolset = 'clang'
class yggdrasil.drivers.CModelDriver.MSVCArchiver(**kwargs)[source]

Bases: ArchiverBase

Microsoft Visual Studio C Archiver.

compatible_toolsets = ['msvc', 'llvm']
default_executable = 'LIB'
default_flags = []
is_gnu = False
languages = ['c', 'c++']
library_ext = '.a'
output_key = '/OUT:%s'
platforms = ['Windows']
search_path_envvar = ['LIB']
shared_library_flag = None
static_library_flag = None
toolname = 'LIB'
toolset = 'msvc'
class yggdrasil.drivers.CModelDriver.MSVCCompiler(**kwargs)[source]

Bases: CCompilerBase

Microsoft Visual Studio C Compiler.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

combine_with_linker = True
compatible_toolsets = ['msvc']
default_archiver = 'LIB'
default_executable = 'cl'
default_flags = ['/W4', '/Zi', '/FS', '/EHsc', '/TP', '/nologo', '-D_CRT_SECURE_NO_WARNINGS']
default_linker = 'LINK'
is_gnu = False
linker_attributes = {'default_executable': None, 'default_executable_env': 'LINK', 'default_flags_env': 'LDFLAGS', 'flag_options': {'library_dirs': '/LIBPATH:%s', 'library_libs': '', 'library_libs_nonstd': ''}, 'output_first': True, 'output_first_library': False, 'output_key': '/OUT:%s', 'search_path_envvar': ['LIB'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);'], 'shared_library_flag': '/DLL'}
linker_switch = '/link'
output_first = True
output_key = '/Fo%s'
platforms = ['Windows']
product_exts = ['.dir', '.ilk', '.pdb', '.sln', '.vcxproj', '.vcxproj.filters', '.exp', '.lib']
search_path_envvar = ['INCLUDE']
search_path_flags = None
classmethod tool_version(**kwargs)[source]

Determine the version of this tool.

Parameters

**kwargs – Keyword arguments are passed to cls.call.

Returns

Version of the tool.

Return type

str

toolname = 'cl'
toolset = 'msvc'
version_flags = []
yggdrasil.drivers.CModelDriver.get_OSX_SYSROOT()[source]

Determin the path to the OSX SDK.

Returns

Full path to the SDK directory if one is located. None

otherwise.

Return type

str

yggdrasil.drivers.CPPModelDriver module

class yggdrasil.drivers.CPPModelDriver.CPPCompilerBase(**kwargs)[source]

Bases: CCompilerBase

Base class for C++ compilers.

classmethod add_standard_flag(flags)[source]

Add a standard flag to the list of flags.

Parameters

flags (list) – Compilation flags.

cpp_std = 'c++14'
default_executable = None
default_executable_env = 'CXX'
default_flags_env = 'CXXFLAGS'
default_linker = None
languages = ['c++']
search_path_flags = ['-E', '-v', '-xc++', '/dev/null']
class yggdrasil.drivers.CPPModelDriver.CPPModelDriver(*args, **kwargs)[source]

Bases: CModelDriver

Class for running C++ models.

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration.

base_languages = ['c']
cfg = <yggdrasil.config.YggConfigParser object>
default_archiver = 'ar'
default_archiver_flags = None
default_compiler = 'g++'
default_compiler_flags = None
default_linker = 'g++'
default_linker_flags = None
dont_declare_channel = True
classmethod finalize_function_io(direction, x)[source]

Finalize info for an input/output channel following function parsing.

Parameters
  • direction (str) – Direction of channel (‘input’ or ‘output’)

  • x (dict) – Channel info.

function_param = {'and': '&&', 'assign': '{name} = {value};', 'assign_copy': 'memcpy({name}, {value}, {N}*sizeof({native_type}));', 'block_end': '}', 'break': 'break;', 'comment': '//', 'copy_any': '{name} = copy_generic({value});', 'copy_array': '{name} = copy_json_array({value});', 'copy_class': '{name} = copy_python({value});', 'copy_function': '{name} = copy_python({value});', 'copy_instance': '{name} = copy_generic({value});', 'copy_obj': '{name} = copy_obj({value});', 'copy_object': '{name} = copy_json_object({value});', 'copy_ply': '{name} = copy_ply({value});', 'copy_schema': '{name} = copy_schema({value});', 'declare': '{type_name} {variable};', 'error': 'throw "{error_msg}";', 'exec_begin': 'int main() {', 'exec_end': '  return 0;\n}', 'exec_prefix': '#include <iostream>\n#include <exception>\n', 'false': 'false', 'false_flag': '0', 'flag_cond': '{flag_var} >= 0', 'for_begin': 'for ({iter_var} = {iter_begin}; {iter_var} < {iter_end}; {iter_var}++) {{', 'fprintf': 'printf("{message}\\n", {variables});', 'free': 'if ({variable} != NULL) {{ free({variable}); {variable} = NULL; }}', 'free_any': 'free_generic({variable});', 'free_array': 'free_json_array({variable});', 'free_class': 'destroy_python({variable});', 'free_function': 'destroy_python({variable});', 'free_instance': 'free_generic({variable});', 'free_obj': 'free_obj({variable});', 'free_object': 'free_json_object({variable});', 'free_ply': 'free_ply({variable});', 'free_schema': 'free_schema({variable});', 'function_def_begin': '{output_type} {function_name}({input_var}) {{', 'function_def_regex': '(?P<flag_type>.+?)\\s*{function_name}\\s*\\((?P<inputs>(?:[^{{\\&])*?)(?:,\\s*(?P<outputs>(?:\\s*(?:[^\\s\\&]+)(?:(?:\\&\\s+)|(?:\\s+(?:\\()?\\&))(?:[^{{])+)+))?\\)\\s*\\{{(?P<body>(?:.*?\\n?)*?)(?:(?:return +(?P<flag_var>.+?)?;(?:.*?\\n?)*?\\}})|(?:\\}}))', 'if_begin': 'if ({cond}) {{', 'if_elif': '}} else if ({cond}) {{', 'if_else': '}} else {{', 'import': '#include "{filename}"', 'indent': '  ', 'index': '{variable}[{index}]', 'init_any': 'init_generic()', 'init_array': 'init_json_array()', 'init_class': 'init_python()', 'init_function': 'init_python()', 'init_instance': 'init_generic()', 'init_obj': 'init_obj()', 'init_object': 'init_json_object()', 'init_ply': 'init_ply()', 'init_schema': 'init_schema()', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray_arr("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': 'YggInput {channel}("{channel_name}", {channel_type});', 'inputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(\\((?P<name_ptr>\\*+)?)?(?P<name>.+?)(?(4)(?:\\)))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'interface': '#include "{interface_library}"', 'line_end': ';', 'not': '!', 'not_flag_cond': '{flag_var} < 0', 'null': 'NULL', 'output': 'YggOutput {channel}("{channel_name}", {channel_type});', 'outputs_def_regex': '\\s*(?P<native_type>(?:[^\\s])+)(\\s+)?(\\()?(?P<ref>\\&)(?(1)(?:\\s*)|(?:\\s+))(?P<name>.+?)(?(2)(?:\\)|(?:)))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'print': 'std::cout << "{message}" << std::endl;', 'print_any': 'display_generic({object});', 'print_array': 'display_json_array({object});', 'print_bytes': 'printf("%s\\n", {object});', 'print_class': 'display_python({object});', 'print_complex': 'print_complex({object});', 'print_float': 'printf("%f\\n", {object});', 'print_function': 'display_python({object});', 'print_generic': 'std::cout << {object} << std::endl << std::flush;', 'print_instance': 'display_generic({object});', 'print_int': 'printf("%i\\n", {object});', 'print_obj': 'display_obj({object});', 'print_object': 'display_json_object({object});', 'print_ply': 'display_ply({object});', 'print_schema': 'display_schema({object});', 'print_string': 'printf("%s\\n", {object});', 'print_uint': 'printf("%u\\n", {object});', 'print_unicode': 'printf("%s\\n", {object});', 'quote': '"', 'recv_function': '{channel}.recvRealloc', 'recv_heap': '{channel}.recvRealloc', 'recv_stack': '{channel}.recv', 'return': 'return {output_var};', 'send_function': '{channel}.send', 'true': 'true', 'true_flag': '1', 'try_begin': 'try {', 'try_error_type': 'const std::exception&', 'try_except': '}} catch ({error_type} {error_var}) {{', 'vector_regex': '(?:std\\:\\:)?vector\\<\\s*(?P<type>.*)\\s*\\>', 'while_begin': 'while ({cond}) {{'}
classmethod get_json_type(native_type)[source]

Get the JSON type from the native language type.

Parameters

native_type (str) – The native language type.

Returns

The JSON type.

Return type

str, dict

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class.

Returns

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type

dict

include_arg_count = True
include_channel_obj = False
interface_library = 'ygg++'
interface_map = {'call': 'flag = {channel_obj}.call({nargs}, {outputs}, {input_refs})', 'client': 'YggRpcClient {channel_obj}("{channel_name}", {datatype_out}, {datatype_in})', 'import': '#include "YggInterface.hpp"', 'input': 'YggInput {channel_obj}("{channel_name}")', 'output': 'YggOutput {channel_obj}("{channel_name}")', 'recv': 'flag = {channel_obj}.recv({nargs}, {input_refs})', 'send': 'flag = {channel_obj}.send({nargs}, {outputs})', 'server': 'YggRpcServer {channel_obj}("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'YggTimesync {channel_obj}("{channel_name}", "{time_units}")'}
internal_libraries = {'datatypes': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/datatypes', 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': [], 'internal_dependencies': ['regex'], 'language': 'c++', 'libtype': 'object'}, 'python_wrapper': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C', 'external_dependencies': ['python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'language': 'c', 'libtype': 'shared', 'linker_language': 'c', 'source': 'python_wrapper.c'}, 'regex': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_posix': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_win32': {'directory': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c++', 'libtype': 'object', 'source': 'regex_win32.cpp'}, 'ygg++': {'compiler_flags': ['-fPIC'], 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/CPP'], 'internal_dependencies': ['regex', 'datatypes', 'python_wrapper'], 'language': 'c++', 'linker_language': 'c++', 'source': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/CPP/YggInterface.cpp'}}
inverse_type_map = None
classmethod is_vector(var)[source]

Determine if a variable uses a vector.

Parameters

var (dict) – Variable.

Returns

True if it is a vector, False otherwise.

Return type

bool

language = 'c++'
language_aliases = ['cpp', 'cxx']
language_ext = ['.cpp', '.CPP', '.cxx', '.C', '.c++', '.cc', '.cp', '.tcc', '.hpp', '.HPP', '.hxx', '.H', '.h++', '.hh', '.hp', '.h']
classmethod output2input(var, in_definition=True)[source]

Perform conversion necessary to turn an output variable into an corresponding input that can be used to format a function definition.

Parameters
  • var (dict) – Variable definition.

  • in_definition (bool, optional) – If True, the returned dictionary corresponds to an input variable in a function definition. If False, the returned value will correspond to an input to a function. Defaults to True.

Returns

Updated variable definition.

Return type

dict

classmethod requires_length_var(var)[source]

Determine if a variable requires a separate length variable.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if a length variable is required, False otherwise.

Return type

bool

classmethod set_env_class(**kwargs)[source]

Set environment variables that are instance independent.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.

Returns

Environment variables for the model process.

Return type

dict

classmethod write_declaration(var, **kwargs)[source]

Return the lines required to declare a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines declaring the variable.

Return type

list

classmethod write_model_recv(channel, recv_var, **kwargs)[source]

Write a model receive call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being received from was stored in.

  • recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a receive call in this language.

Return type

list

classmethod write_model_send(channel, send_var, **kwargs)[source]

Write a model send call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being sent to was stored in.

  • send_var (dict, list) – Information on one or more variables containing information that will be sent.

  • flag_var (str, optional) – Name of flag variable that the flag should be stored in. Defaults to ‘flag’,

  • allow_failure (bool, optional) – If True, the returned lines will call a break if the flag is False. Otherwise, the returned lines will issue an error. Defaults to False.

Returns

Lines required to carry out a send call in this language.

Return type

list

classmethod write_try_except(try_contents, except_contents, error_var='e', error_type=None, **kwargs)[source]

Return the lines required to complete a try/except block.

Parameters
  • try_contents (list) – Lines of code that should be executed inside the try block.

  • except_contents (list) – Lines of code that should be executed inside the except block.

  • error_var (str, optional) – Name of variable where the caught error should be stored. Defaults to ‘e’. If ‘…’, the catch clause will catch all errors, but there will not be a name error.

  • error_type (str, optional) – Name of error type that should be caught. If not provided, defaults to None and will be set based on the class function_param entry for ‘try_error_type’. If ‘…’, the catch clause will catch all errors and error_var will be ignored.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines of code perfoming a try/except block.

class yggdrasil.drivers.CPPModelDriver.ClangPPCompiler(**kwargs)[source]

Bases: CPPCompilerBase, ClangCompiler

Interface class for clang++ compiler.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

default_executable = 'clang++'
default_flags = ['-g', '-Wall']
default_linker = 'clang++'
classmethod get_executable_command(args, skip_flags=False, unused_kwargs=None, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • **kwargs – Additional keyword arguments will be passed to the parent class’s method.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(skip_standard_flag=False, **kwargs)[source]

Get a list of compiler flags.

Parameters
  • skip_standard_flag (bool, optional) – If True, the C++ standard flag will not be added. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

is_gnu = False
is_linker = False
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);']}
toolname = 'clang++'
class yggdrasil.drivers.CPPModelDriver.ClangPPLinker(**kwargs)[source]

Bases: ClangLinker

Interface class for clang++ linker (calls to ld).

default_executable = 'clang++'
default_flags = []
is_gnu = False
languages = ['c++']
library_ext = '.so'
toolname = 'clang++'
toolset = 'llvm'
class yggdrasil.drivers.CPPModelDriver.GPPCompiler(**kwargs)[source]

Bases: CPPCompilerBase, GCCCompiler

Interface class for G++ compiler/linker.

aliases = ['gnu-c++']
default_executable = 'g++'
default_flags = ['-g', '-Wall']
default_linker = 'g++'
classmethod get_flags(skip_standard_flag=False, **kwargs)[source]

Get a list of compiler flags.

Parameters
  • skip_standard_flag (bool, optional) – If True, the C++ standard flag will not be added. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

is_gnu = True
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'flag_options': {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}, 'library_rpath': '-Wl,-rpath'}, 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);']}
toolname = 'g++'
class yggdrasil.drivers.CPPModelDriver.MSVCPPCompiler(**kwargs)[source]

Bases: CPPCompilerBase, MSVCCompiler

Inteface class for MSVC compiler when compiling C++.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

default_archiver = 'LIB'
default_executable = 'cl'
default_flags = ['/W4', '/Zi', '/FS', '/EHsc', '/TP', '/nologo', '-D_CRT_SECURE_NO_WARNINGS']
default_linker = 'LINK'
dont_create_linker = True
is_gnu = False
linker_attributes = {'default_executable': None, 'default_executable_env': 'LINK', 'default_flags_env': 'LDFLAGS', 'flag_options': {'library_dirs': '/LIBPATH:%s', 'library_libs': '', 'library_libs_nonstd': ''}, 'output_first': True, 'output_first_library': False, 'output_key': '/OUT:%s', 'search_path_envvar': ['LIB'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);'], 'shared_library_flag': '/DLL'}
search_path_flags = None
toolname = 'cl++'

yggdrasil.drivers.CompiledModelDriver module

class yggdrasil.drivers.CompiledModelDriver.ArchiverBase(**kwargs)[source]

Bases: LinkerBase

Base class for archivers.

static_library_flag

Flag that should be prepended to the archiver tool arguments to indicated that a static library should be produced.

Type

str

library_name_key

Option key indicating the name of a library that should be linked against.

Type

str

library_directory_key

Option key indicating a directory that should be included in the linker search path for libraries.

Type

str

library_prefix

Prefix that should be added to library paths.

Type

str

library_ext

Extension that should be used for static libraries.

Type

str

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

flag_options = {}
classmethod get_flags(**kwargs)[source]

Get a list of flags for this archiver tool.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Archiver flags.

Return type

list

classmethod get_output_file(obj, **kwargs)[source]

Determine the appropriate output file that will result when linking a given object file.

Parameters
  • obj (str) – Object file being linked that name base will be taken from.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Full path to file that will be produced.

Return type

str

library_ext = None
static_library_flag = '-static'
tooltype = 'archiver'
class yggdrasil.drivers.CompiledModelDriver.BuildToolBase(**kwargs)[source]

Bases: CompilerBase

Base class for build tools which are used to coordinate compilation.

Parameters
  • buildfile (str, optional) – File containing information about the build. Defaults to default_buildfile class attribute, if set.

  • builddir (str, optional) – Directory where build files should be placed. Defaults to directory where build is called.

  • sourcedir (str, optional) – Directory where source files are stored. Defaults to directory where build is called.

  • target (str, optional) – Build target. If not provided, build will be created without a target.

Class Attributes:

default_buildfile = None
flag_options = {}
tooltype = 'buildtool'
class yggdrasil.drivers.CompiledModelDriver.CompilationToolBase(**kwargs)[source]

Bases: object

Base class for compilation command line tools.

Class Attributes:
toolname (str): Tool name used for registration and as a default for the

executable. [REQUIRED]

aliases (list): Alternative names that the tool might have. tooltype (str): Tool type. One of ‘compiler’, ‘linker’, or ‘archiver’.

[AUTOMATED]

languages (list): Programming languages that this tool can be used on.

[REQUIRED]

platforms (list): Platforms that the tool is available on. Defaults to

[‘Windows’, ‘MacOS’, ‘Linux’].

default_executable (str): The default tool executable command if

different than the toolname.

default_executable_env (str): Environment variable where the executable

command might be stored.

default_flags (list): Default flags that should be used when calling the

tool (e.g. for verbose output or enhanced warnings).

default_flags_env (str): Environment variable where default flags for

the tools might be stored.

output_key (str): Option key indicating the output file name. output_first (bool): If True, the output key (and its value) are put in

front of the other flags.

flag_options (collections.OrderedDict): Mappings between keyword

arguments passed to get_flags and command line tool flags. Keys indicate the keyword argument that will be processed in Python and values are the string used with the tool on the command line to indicate the desired flag. Flags that contain ‘%s’ will be formatted with values passed to get_flags as the designated keyword argument. Flags that do not contain ‘%s’ will assumed to act on the following argument. If the value passed to get_flags under the designated keyword argument is not a boolean, it will be appended to the flag list following the corresponding flag. If the value is a boolean and it is True, only the flag will be added to the list of flags. The order of entries indicates the order the flags should be added to the list.

search_path_envvar (list): Environment variables containing a list of paths

to search for library files. Either search_path_envvar or search_path_flags must be set. [REQUIRED]

search_path_env (list): Paths relative to the env prefix that should

be searched if the VIRTUAL_ENV or CONDA_PREFIX environment variable is set.

search_path_flags (list): Flags that should be passed to the tool

executable in order to locate the search path. Either search_path_envvar or search_path_flags must be set. [REQUIRED]

search_regex_begin (str): Search string indicating where the set of

paths begins in the output from running the tool executable with the search_path_flags. If None, the search is performed from the very beginning of the output.

search_regex_end (str): Search string indicating where the set of

paths ends in the output from running the tool executable with the search_path_flags. If None, the search is performed to the very end of the output.

search_regex (list): Regex strings that should be used to locate paths

in the output from running the tool executable with the search_path_flags.

product_exts (list): List of extensions that will be added to the

output file’s base to get a list of products that might be produced by calling the compilation tool.

product_files (list): List of file basenames that will be joined with

the directory containing each output to get a list of products that might be produced by calling the compilation tool.

remove_product_exts (list): List of extensions or directories matching

entries in product_exts and product_files that should be removed during cleanup. Be careful when adding files to this list.

aliases = []
classmethod append_flags(out, key, value, **kwargs)[source]

Append one ore more flag(s) to a list of flags based on a key value/set.

Parameters
  • out (list) – Existing list that the new flag should be appended to.

  • key (str) – String identifying the type of flag. See create_flag for additional details.

  • value (object) – Value that should be set in the flag. If a list, multiple flags are added, one for each item in the list.

  • prepend (bool, optional) – If True, new flags are prepended to the front of the list rather than the end. Defaults to False. This keyword argument is ignoerd if position is provided.

  • position (int, optional) – If not None, this is the position that the new elements should be added to the existing flags. Positive positions indicate indexes into the existing list of flags. Negative are relative to the end of the existing list such that -1 is the same as just appending the new flags to the end of the list. Defaults to None and prepend takes precedence.

  • no_duplicates (bool, optional) – If True, the new flags being added will be checked against the existing ones to ensure that there are not duplicates. If False, the new flags are added reguardless of the existing flags. Defaults to False.

  • allow_duplicate_values (bool, optional) – If True, the same key can be added with the same value more than once. Otherwise, only the first instance of the value is added. Defaults to False.

Raises
  • ValueError – If there are unexpected keyword arguments.

  • ValueError – If no_duplicates is True and the existing list of flags already contains a flag matching the provided flag key.

classmethod append_product(products, src, new, new_dir=None, dont_append_src=False)[source]

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters
  • products (list) – List of of existing products that new product should be appended to.

  • src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).

  • new (str) – New product that should be appended to the list.

  • new_dir (str, optional) – Directory that should be used as base when adding files listed in cls.product_files. Defaults to os.path.dirname(new).

  • dont_append_src (bool, optional) – If True and src is in the list of products, it will be removed. Defaults to False.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

classmethod call(args, language=None, toolname=None, skip_flags=False, dry_run=False, out=None, overwrite=False, products=None, allow_error=False, working_dir=None, additional_args=None, suffix='', **kwargs)[source]

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters
  • args (list) – The arguments that should be passed to the tool.

  • language (str, optional) – Language of tool that should be used. If different than the languages supported by the current tool, the correct tool is used instead. Defaults to None and is ignored.

  • toolname (str, optional) – Name of compilation tool that should be used. Defaults to None and the default tool for the language will be used.

  • skip_flags (bool, optional) – If True, args is assumed to include any necessary flags. If False, args are assumed to the files that the tool is called on and flags are determined from them. Defaults to False.

  • dry_run (bool, optional) – If True, the tool won’t be called, but the products will be updated. Defautls to False.

  • out (str, optional) – Full path to output file that should be created. If None, the path will be determined from the path to the first argument provided. Defaults to None. This keyword argument will be ignored if skip_flags is True.

  • overwrite (bool, optional) – If True, the existing compile file will be overwritten. Otherwise, it will be kept and this function will return without recompiling the source file.

  • products (list, optional) – Existing Python list that additional products produced by the compilation should be appended to. Defaults to None and is ignored.

  • allow_error (bool, optional) – If True and there is an error when call the executable, it will be ignored. If False, errors will result in an exception being raised. Defaults to False.

  • working_dir (str, optional) – Working directory where tool should be called from. This will also be used to construct the path for the output file. Defaults to None and is ignored.

  • additional_args (list, optional) – Additional arguments that should be appended to args before continuing. Defaults to None and is ignored.

  • suffix (str, optional) – Suffix that should be added to the output file (before the extension). Defaults to “”.

  • **kwargs – Additional keyword arguments are passed to cls.get_executable_command. and tools.popen_nobuffer.

Returns

Output to stdout from the command execution if skip_flags is

True, produced file otherwise.

Return type

str

Raises

RuntimeError – If there is an error when running the command and allow_error is False.

compatible_toolsets = []
classmethod create_flag(key, value)[source]

Create a flag from a key/value set.

Parameters
  • key (str) – String identifying the type of flag. If key contains ‘%s’, it is assumed that the flag will be produced by formatting the value according to key. If key dosn’t contain ‘%s’, it is assumed that key indicates that value is the following item and they both will be returned. If key is an empty string, then the value is assumed to constitute the entire flag.

  • value (object) – Value that should be set in the flag. If a list, multiple flags are returned, one for each item in the list.

Returns

Items representing the flag.

Return type

list

default_executable = None
default_executable_env = None
default_flags = []
default_flags_env = None
classmethod env_matches_tool()[source]

Determine if the executable pointed to by any environment variable matches this compilation tool.

Returns

True if the environment variable matches, False otherwise.

Return type

bool

classmethod file2base(fname)[source]

Determine basename from path.

Parameters

fname (str) – Full or partial path to file.

Returns

File name without extension.

Return type

str

flag_options = {}
classmethod get_alternate_class(toolname=None, language=None)[source]

Return an alternate class to use if the provided toolname dosn’t match the current tool.

Parameters
  • toolname (str, optional) – Name of compilation tool that should be used. Defaults to None and the current toolname is assumed.

  • language (str, optional) – Language that alternate class should support. Defaults to None and the current language will be assumed.

Returns

The compilation tool that corresponds

to the provided toolname.

Return type

CompilationToolBase

classmethod get_env_flags()[source]

Get a list of flags stored in the environment variables.

Returns

Flags for the tool.

Return type

list

classmethod get_env_prefixes()[source]

Determine the virtualenv/conda path prefixes.

Returns

Virtualenv/conda path prefixes. Empty list will be

returned if virtualenv/conda are not active.

Return type

list

classmethod get_executable(full_path=False)[source]

Determine the executable that should be used to call this tool.

Parameters

full_path (bool, optional) – If True the full path to the executable file will be returned. Defaults to False.

Returns

Name of (or path to) the tool executable.

Return type

str

classmethod get_executable_command(args, skip_flags=False, unused_kwargs=None, use_ccache=False, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • skip_flags (bool, optional) – If True, args is assumed to include any necessary flags. If False, args are assumed to the files that the tool is called on and flags are determined from them. Defaults to False.

  • unused_kwargs (dict, optional) – Existing Python dictionary that unused keyword arguments will be added to. Defaults to None and is initialized to an empty dict.

  • use_ccache (bool, optional) – If True, ccache will be added to the compilation executable. Defaults to False.

  • **kwargs – Additional keyword arguments are ignored and stored in unused_kwargs if provided.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(flags=None, outfile=None, output_first=None, unused_kwargs=None, skip_defaults=False, dont_skip_env_defaults=False, remove_flags=None, **kwargs)[source]

Get a list of flags for the tool.

Parameters
  • flags (list, optional) – User defined flags that should be included. Defaults to empty list.

  • outfile (str, optional) – If provided, it is appended to the end of the flags following the cls.output_key flag to indicate that this is the name of the output file. Defaults to None and is ignored.

  • output_first (bool, optional) – If True, output flag(s) will be placed at the front of the returned flags. If False, they are placed at the end. Defaults to None and is set by cls.output_first.

  • unused_kwargs (dict, optional) – Existing dictionary that unused keyword arguments should be added to. Defaults to None and is ignored.

  • skip_defaults (bool, optional) – If True, the default flags will not be added. Defaults to False.

  • dont_skip_env_defaults (bool, optional) – If skip_defaults is True, and this keyword is True, the flags from the environment variable will be added. Defaults to False.

  • remove_flags (list, optional) – List of flags to remove. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are ignored and added to unused_kwargs if provided.

Returns

Flags for the tool.

Return type

list

classmethod get_language_ext()[source]

Get the extensions associated with the language that this tool can handle.

Returns

Language file extensions.

Return type

list

classmethod get_search_path(env_only=False, libtype=None, cfg=None)[source]

Determine the paths searched by the tool for external library files.

Parameters
  • env_only (bool, optional) – If True, only the search paths as indicated by a virtualenv/conda environment are returned. Defaults to False.

  • libtype (str, optional) – Library type being searched for. Defaults to None.

  • cfg (YggConfigParser, optional) – Configuration object currently being updated. Defaults to the global configuration.

Returns

List of paths that the tools will search.

Return type

list

classmethod get_source_products(products, source_products=[])[source]

Get the list of products that should be removed without checking for source files based on cls.remove_product_exts.

Parameters
  • products (list) – List of products that should be checked against cls.remove_product_exts.

  • source_products (list, optional) – Existing list of products that new source_products should be added to. Defaults to [].

Returns

Products that should be removed without checking for source

files.

Return type

list

classmethod get_tool_suffix()[source]

Get the string that should be added to tool products based on the tool used.

Returns

Suffix that should be added to tool products to indicate the

tool used.

Return type

str

is_build_tool = False
is_gnu = False
classmethod is_installed()[source]

Determine if this tool is installed by looking for the executable.

Returns

True if the tool is installed, False otherwise.

Return type

bool

languages = []
output_first = False
output_key = '-o'
platforms = ['Windows', 'MacOS', 'Linux']
product_exts = []
product_files = []
remove_product_exts = []
classmethod remove_products(src, new)[source]

Remove products produced during compilation.

Parameters
  • src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).

  • new (str) – The full path to the primary product of the compilation call (usually an object, executable, or library).

search_path_env = None
search_path_envvar = None
search_path_flags = None
search_regex = ['([^\n]+)']
search_regex_begin = None
search_regex_end = None
classmethod set_env(existing=None, **kwargs)[source]

Set environment variables required for compilation.

Parameters
  • existing (dict, optional) – Existing dictionary of environment variables that new variables should be added to. Defaults to a copy of os.environ.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Environment variables for the model process.

Return type

dict

source_product_exts = []
tool_suffix_format = '_%sx'
toolname = None
toolset = None
tooltype = None
version_flags = ['--version']
classmethod write_wrappers(**kwargs)[source]

Write any wrappers needed to compile and/or run a model.

Parameters

**kwargs – Keyword arguments are ignored (only included to allow cascade from child classes).

Returns

Full paths to any created wrappers.

Return type

list

class yggdrasil.drivers.CompiledModelDriver.CompilationToolMeta(name, bases, class_dict)[source]

Bases: type

Meta class for registering compilers.

class yggdrasil.drivers.CompiledModelDriver.CompiledModelDriver(*args, **kwargs)[source]

Bases: ModelDriver

Base class for models written in compiled languages.

Parameters
  • name (str) – Driver name.

  • args (str or list) – The model executable and any arguments that should be passed to the model executable.

  • source_files (list, optional) – Source files that should be compiled into an executable. Defaults to an empty list and the driver will search for a source file based on the model executable (the first model argument).

  • compiler (str, optional) – Command or path to executable that should be used to compile the model. If not provided, the compiler will be determined based on configuration options for the language (if present) and the registered compilers that are available on the current operating system.

  • compiler_flags (list, optional) – Flags that should be passed to the compiler during compilation. If nto provided, the compiler flags will be determined based on configuration options for the language (if present), the compiler defaults, and the default_compiler_flags class attribute.

  • linker (str, optional) – Command or path to executable that should be used to link the model. If not provided, the linker will be determined based on configuration options for the language (if present) and the registered linkers that are available on the current operating system

  • linker_flags (list, optional) – Flags that should be passed to the linker during compilation. If nto provided, the linker flags will be determined based on configuration options for the language (if present), the linker defaults, and the default_linker_flags class attribute.

  • **kwargs – Additional keyword arguments are passed to parent class.

Class Attributes:
default_compiler (str): Name of compiler that will be used if not

set explictly by instance or config file.

default_compiler_flags (list): Flags that should be passed to the

compiler by default for this language.

default_linker (str): Name of linker that will be used if not

set explictly by instance or config file

default_linker_flags (list): Flags that should be passed to the

linker by default for this language.

allow_parallel_build (bool): If True, a file can be compiled by

two processes simultaneously. If False, it cannot and an MPI barrier will be used to prevent simultaneous compilation. Defaults to False.

source_files

Source files.

Type

list

compiler

Name or path to the compiler that should be used.

Type

str

compiler_flags

Compiler flags.

Type

list

linker

Name or path to the linker that should be used.

Type

str

linker_flags

Linker flags.

Type

list

compiler_tool

Compiler tool that will be used.

Type

CompilerBase

linker_tool

Linker tool that will be used.

Type

LinkerBase

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

allow_parallel_build = False
buildfile_locked(dry_run=False)[source]

Context manager for locked build file.

classmethod call_compiler(src, language=None, toolname=None, dont_build=None, **kwargs)[source]

Compile a source file into an executable or linkable object file, checking for errors.

Parameters
  • src (str) – Full path to source file.

  • out (str, optional) – Full path to the output object file that should be created. Defaults to None and is created from the provided source file.

  • flags (list, optional) – Compilation flags. Defaults to []. If compiler_flags is present, flags is replaced by compiler_flags.

  • compiler_flags (list, optional) – Alternative to flags. Ignored if not provided.

  • dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.

  • dont_build (bool, optional) – If True, cmake configuration/generation will be run, but the project will not be built. Defaults to None. If provided, this overrides dont_link.

  • overwrite (bool, optional) – If True, the existing compile file will be overwritten. Otherwise, it will be kept and this function will return without recompiling the source file.

  • language (str, optional) – Language that should be used to compile the files. Defaults to None and the language of the current driver is used.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • products (list, optional) – Existing Python list that additional products produced by the compilation should be appended to. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are passed to run_executable. and call_linker if dont_link is False.

Returns

Full path to compiled source.

Return type

str

Raises
  • RuntimeError – If there is an error in calling the compiler.

  • RuntimeError – If the compilation command dosn’t yield the specified output file.

classmethod call_linker(obj, language=None, toolname=None, **kwargs)[source]

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters
  • obj (list) – Object files that should be linked.

  • language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • **kwargs – Additional keyword arguments are passed to run_executable.

Returns

Full path to compiled source.

Return type

str

classmethod cleanup_dependencies(products=None, verbose=False, **kwargs)[source]

Cleanup dependencies.

classmethod compile_dependencies(toolname=None, dep=None, **kwargs)[source]

Compile any required internal libraries, including the interface.

compile_dependencies_instance(*args, **kwargs)[source]

Compile dependencies specifically for this instance.

compile_model(source_files=None, skip_interface_flags=False, **kwargs)[source]

Compile model executable(s).

Parameters
  • source_files (list, optional) – Source files that will be compiled. Defaults to None and is set to the source_files attribute.

  • skip_interface_flags (bool, optional) – If True, interface flags will not be added. This includes the logger flag specifying the current logging level. Defaults to False.

  • **kwargs – Keyword arguments are passed on to the call_compiler method.

Returns

Compiled model file path.

Return type

str

classmethod configure(cfg, **kwargs)[source]

Add configuration options for this language.

Parameters
  • cfg (CisConfigParser) – Config class that options should be set for.

  • **kwargs – Additional keyword arguments are used to set tool configuration options (e.g. ‘compiler’).

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

classmethod configure_executable_type(cfg)[source]

Add configuration options specific in the executable type before the libraries are configured.

Parameters

cfg (CisConfigParser) – Config class that options should be set for.

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

classmethod configure_libraries(cfg)[source]

Add configuration options for external libraries in this language.

Parameters

cfg (CisConfigParser) – Config class that options should be set for.

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

classmethod configure_library(cfg, k)[source]

Add configuration options for an external library.

Parameters

cfg (YggConfigParser) – Config class that options should be set for.

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

default_archiver = None
default_archiver_flags = None
default_compiler = None
default_compiler_flags = None
default_linker = None
default_linker_flags = None
classmethod executable_command(args, exec_type='compiler', toolname=None, **kwargs)[source]

Compose a command for running a program using the compiler for this language and the provied arguments. If not already present, the compiler command and compiler flags are prepended to the provided arguments.

Parameters
  • args (list) – The program that returned command should run and any arguments that should be provided to it. For the compiler, this means the source files, for the linker, this means the object files.

  • exec_type (str, optional) – Type of executable command that will be returned. If ‘compiler’, a command using the compiler is returned, if ‘linker’, a command using the linker is returned, and if ‘direct’, the raw args being provided are returned. Defaults to ‘compiler’.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • **kwargs – Additional keyword arguments are passed to either get_linker_flags or get_compiler_flags.

Returns

Arguments composing the command required to run the program

from the command line using the compiler for this language.

Return type

list

Raises

ValueError – If exec_type is not ‘compiler’, ‘linker’, or ‘direct’.

executable_type = 'compiler'
classmethod get_available_tools(tooltype)[source]

Return the registry of compilation tools for this language.

Parameters

tooltype (str) – Name of tool type that should be returned. Values include ‘compiler’, ‘linker’, or ‘archiver’.

Returns

Registry of tools for this language.

Return type

dict

classmethod get_buildfile_lock(fname=None, context=None, instance=None, **kwargs)[source]

Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

Parameters
  • name (str) – Build file.

  • context (threading.Context) – Threading context.

  • instance (ModelDriver) – Driver instance that should be used.

  • **kwargs – Additional keyword arguments are passed to the FileLock initialization.

Returns

Lock for the buildfile.

Return type

FileLock

classmethod get_compiler_flags(toolname=None, compiler=None, **kwargs)[source]

Determine the flags required by the current compiler.

Parameters
  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • compiler (CompilerBase, optional) – Compilation tool class for the compiler that should be used. Defaults to None and is set based on toolname.

  • **kwargs – Keyword arguments are passed to cls.update_compiler_kwargs first and then the compiler’s get_flags method.

Returns

Flags for the compiler.

Return type

list

classmethod get_dependency_include_dirs(dep, toolname=None, default=None)[source]

Get the include directories for a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the library.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns

Full paths to the directories containing the dependency’s

header(s).

Return type

list

Raises

ValueError – If the include directory cannot be determined for the specified dependency and default is None.

classmethod get_dependency_info(dep, toolname=None, default=None)[source]

Get the dictionary of information associated with a dependency.

Parameters
  • dep (str) – Name of internal or external dependency.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • default (dict, optional) – Information dictionary that should be returned if dep cannot be located. Defaults to None and an error will be raised if dep cannot be found.

Returns

Dependency info.

Return type

dict

classmethod get_dependency_library(dep, default=None, libtype=None, commtype=None, toolname=None)[source]

Get the library location for a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the library.

  • default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

  • libtype (str, optional) – Library type that should be returned. Valid values are ‘static’ and ‘shared’. Defaults to None and will be set on the dependency’s libtype (if it has one) or the _default_libtype parameter (if it doesn’t).

  • commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns

Full path to the library file. For header only libraries,

an empty string will be returned.

Return type

str

Raises
  • ValueError – If libtype is not ‘static’ or ‘shared’.

  • ValueError – If the path to the library cannot be determined for the specified dependency and default is None.

classmethod get_dependency_object(dep, default=None, commtype=None, toolname=None)[source]

Get the location of an object file for a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the object file.

  • default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

  • commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns

Full path to the object file.

Return type

str

classmethod get_dependency_order(deps, toolname=None)[source]

Get the correct dependency order, including any dependencies for the direct dependencies.

Parameters
  • deps (list) – Dependencies in order.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns

Dependency order.

Return type

list

classmethod get_dependency_source(dep, toolname=None, default=None)[source]

Get the path to the library source files (or header files) for a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the library.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns

Full path to the library source file. For header only libraries

this will be the header location.

Return type

str

classmethod get_external_libraries(no_comm_libs=False)[source]

Determine the external libraries that are required based on the default comm.

Parameters

no_comm_libs (bool, optional) – If True, libraries for the installed comms are not included in the returned list. Defaults to False.

Returns

The names of external libraries required by the interface

library, including the dependency libraries for the installed comm libraries.

Return type

list

classmethod get_internal_suffix(commtype=None)[source]

Determine the suffix that should be used for internal libraries.

Parameters

commtype (str, optional) – If provided, this is the communication type that should be used for the model. If None, the default comm is used.

Returns

Suffix that should be added to internal libraries to

differentiate between different dependencies.

Return type

str

classmethod get_linker_flags(toolname=None, **kwargs)[source]

Determine the flags required by the current linker.

Parameters
  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • **kwargs – Keyword arguments are passed to cls.update_linker_kwargs first and then the linker’s get_flags method.

Returns

Flags for the linker.

Return type

list

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class.

Returns

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type

dict

classmethod get_tool(*args, **kwargs)[source]

Get tool from a driver class.

Parameters
  • *args – Arguments are passed to the get_tool_static method.

  • **kwargs – Keyword arguments are passed to the get_tool_static method.

Returns

Class providing an interface to the specified

compilation tool.

Return type

CompilationToolBase

get_tool_instance(*args, **kwargs)[source]

Get tool from a driver instance.

Parameters
  • *args – Arguments are passed to the get_tool_static method.

  • **kwargs – Keyword arguments are passed to the get_tool_static method.

Returns

Class providing an interface to the specified

compilation tool.

Return type

CompilationToolBase

static get_tool_static(cls, tooltype, toolname=None, return_prop='tool', default=False, language=None)[source]

Get the class associated with the specified compilation tool for this language.

Parameters
  • cls (class, instance) – Compiled driver class or instance of compiled driver class to get tool for.

  • tooltype (str) – Type of compilation tool that should be returned.

  • toolname (str, optional) – Name of the tool that should be returned. Defaults to None and the tool name associated with the provided class/instance will be used.

  • return_prop (str, optional) – Value that should be returned. If ‘tool’, the tool is returned. If ‘name’, the tool name is returned. If ‘flags’, the tool flags are returned. Defaults to ‘tool’.

  • default (object, optiona) – Tool that should be returned if one cannot be identified. If False, an error will be raised when a tool cannot be located. Defaults to False.

  • language (str, optional) – Language of tools that should be returned. Defaults to None if not provided.

Returns

Class providing an interface to the specified

compilation tool.

Return type

CompilationToolBase

Raises
classmethod identify_source_files(args=None, working_dir=None, source_files=None, **kwargs)[source]

Determine the source file based on model arguments.

Parameters
  • args (list, optional) – Arguments provided.

  • working_dir (str, optional) – Working directory.

  • source_files (list, optional) – Source files in the model.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Source files.

Return type

list

is_build_tool = False
classmethod is_configured()[source]

Determine if the appropriate configuration has been performed (e.g. installation of supporting libraries etc.)

Returns

True if the language has been configured.

Return type

bool

classmethod is_language_installed()[source]

Determine if the interpreter/compiler for the associated programming language is installed.

Returns

True if the language interpreter/compiler is installed.

Return type

bool

classmethod is_library_installed(lib, cfg=None)[source]

Determine if a dependency is installed by check for the appropriate config options setting the path to the library files.

Parameters
  • lib (str) – Name of the library that should be checked.

  • cfg (CisConfigParser, optional) – Config class that should be checked. Defaults to yggdrasil.config.ygg_cfg if not provided.

Returns

True if the library is installed, False otherwise.

Return type

bool

classmethod is_tool_installed(tooltype)[source]

Determine if a compilation tool of a certain is installed for this language.

Parameters

tooltype (str) – Type of tool to check for. Supported values include ‘compiler’, ‘linker’, & ‘archiver’.

Returns

True if a tool of the specified type is installed.

Return type

bool

classmethod language_executable(toolname=None)[source]

Command required to compile/run a model written in this language from the command line.

Parameters

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns

Name of (or path to) compiler/interpreter executable required

to run the compiler/interpreter from the command line.

Return type

str

classmethod language_version(toolname=None, **kwargs)[source]

Determine the version of this language.

Parameters
  • **kwargs – Keyword arguments are passed to cls.run_executable.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns

Version of compiler/interpreter for this language.

Return type

str

locked_buildfile = None
model_command()[source]

Return the command that should be used to run the model.

Returns

Any commands/arguments needed to run the model from the

command line.

Return type

list

classmethod mpi_partner_cleanup(self)[source]

Actions cleaning up an MPIPartnerModel.

classmethod mpi_partner_init(self)[source]

Actions initializing an MPIPartnerModel.

parse_arguments(args, **kwargs)[source]

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters
  • args (list) – List of arguments provided.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod partner_buildfile_lock(self)[source]

Actions completing buildfile lock on MPIPartnerModels.

run_model(**kwargs)[source]

Run the model. Unless overridden, the model will be run using run_executable.

Parameters

**kwargs – Keyword arguments are passed to run_executable.

set_env(for_compile=False, compile_kwargs=None, toolname=None, **kwargs)[source]

Get environment variables that should be set for the model process.

Parameters
  • for_compile (bool, optional) – If True, environment variables are set that are necessary for compiling. Defaults to False.

  • compile_kwargs (dict, optional) – Keyword arguments that should be passed to the compiler’s set_env method. Defaults to empty dict.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

classmethod set_env_compiler(compiler=None, toolname=None, **kwargs)[source]

Get environment variables that should be set for the compilation process.

Parameters
  • compiler (CompilerBase, optional) – Compiler that set_env shoudl be called for. If not provided, the default compiler for this language will be used.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

set_target_language()[source]

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns

Name of language.

Return type

str

classmethod update_compiler_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, directory=None, include_dirs=None, definitions=None, skip_interface_flags=False, **kwargs)[source]

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters
  • dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.

  • for_api (bool, optional) – If True, flags are added that are required for compiling internal api libraries in this language. This includes external communication libraries. Defaults to False.

  • for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.

  • commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • include_dirs (list, optional) – If provided, each list element will be added as an included directory flag. Defaults to None and is initialized as an empty list.

  • definitions (list, optional) – If provided, each list element will be added as a defintion. Defaults to None and is initialized to an empty list.

  • skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.

  • internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.

  • external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.

  • **kwargs – Additional keyword arguments are passed to the compiler class’s ‘get_flags’ method and get_linker_flags if dont_link is False.

Returns

Keyword arguments for a get_flags method providing compiler

flags.

Return type

dict

classmethod update_linker_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, libtype='object', skip_interface_flags=False, use_library_path_internal=False, **kwargs)[source]

Update keyword arguments supplied to the linker/archiver get_flags method for various options.

Parameters
  • for_api (bool, optional) – If True, flags are added that are required for linking internal api libraries in this language. This includes external communication libraries. Defaults to False.

  • for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.

  • commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • libtype (str, optional) – Library type that should be created by the linker/archiver. Valid values are ‘static’, ‘shared’, or ‘object’. Defaults to ‘object’.

  • skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.

  • libraries (list, optional) – Full paths to libraries that should be linked against. Defaults to an empty list.

  • internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.

  • external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.

  • use_library_path_internal (bool, optional) – If True, internal dependencies are included as full paths. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the linker (or archiver if static is True) ‘get_flags’ method.

Returns

Keyword arguments for a get_flags method providing linker/

archiver flags.

Return type

dict

write_wrappers(**kwargs)[source]

Write any wrappers needed to compile and/or run a model.

Parameters

**kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Full paths to any created wrappers.

Return type

list

class yggdrasil.drivers.CompiledModelDriver.CompilerBase(**kwargs)[source]

Bases: CompilationToolBase

Base class for compilers.

Parameters
  • linker (str, optional) – Name of the linker that should be used for linking compiled objects. Defaults to None if not provided and default_linker will be used.

  • archiver (str, optional) – Name of the archiver that should be used for combining compiled objects into a static library. Defaults to None if not provided and default_archiver will be used.

  • linker_flags (list, optional) – Flags that should be used when linking compiled objects. Defaults to default_linker_flags if not provided.

  • archiver_flags (list, optional) – Flags that should be used for combining compiled objects into a static library. Defaults to default_archiver_flags if not provided.

Class Attributes:
compile_only_flag (str): Flag that should prepended to compiler/linker

combination tool arguments to indicate that only compilation should be performed.

default_linker (str): Name of linker that should be used after compiling

with this compiler. If not set, it is assumed that this compiler is also a linker.

default_archiver (str): Name of archiver that should be used to create

a static library after compiling with this compiler. If not set, it is assumed that this compiler is also a linker.

default_linker_flags (list): Flags that should be used with the linker

if no other flags are provided.

default_archiver_flags (list): Flags that should be used with the

archiver if no other flags are provided.

linker_switch (str): Flag to indicate beginning of flags that should be

passed to the linker from a call to a compiler/linker combination tools (e.g. /link on Windows).

object_ext (str): Extension that should be used for object files. is_linker (bool): If True, the tool also serves as a linker and a

separate linker class will be automatically generating from the linker_attributes dictionary. This will be set to True if no_separate_linking is True.

no_separate_linking (bool): If True, the tool severs as linker but

cannot be called for just compilation or linking alone.

linker_attributes (dict): Attributes that should be added to the linker

class created for this tool if is_linker is True.

linker_base_classes (tuple): Base classes that should be used to create

the default linker from the compiler tool. If None, (LinkerBase, ) is used if no_separate_linking is False and (DummyLinkerBase, ) is used if no_separate_linking is True.

combine_with_linker (bool): If True, the compiler and linker flags can

be combined and passed to the compiler executable to perform both operations in succession. If False, the compilation and linking steps must be performed separately. If None, this is determined by checking if the compiler and linker names match.

classmethod archiver()[source]

Get the associated archiver class.

Returns

Archiver class associated with this compiler.

Return type

ArchiverToolBase

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

classmethod call(args, dont_link=None, skip_flags=False, out=None, libtype=None, additional_objs=None, **kwargs)[source]

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters
  • args (list) – The arguments that should be passed to the tool.

  • dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable or library. Defaults to True if cls.no_separate_linking is True or libtype is ‘object’ and False otherwise.

  • skip_flags (bool, optional) – If True, args is assumed to include any necessary flags. If False, args are assumed to the files that the tool is called on and flags are determined from them. Defaults to False.

  • out (str, optional) – Full path to output file that should be created. If None, the path will be determined from the path to the first arguments provided. Defaults to None. This keyword argument will be ignored if skip_flags is True.

  • libtype (str, optional) – Library type that should be created by the linker/archiver. Defaults to None.

  • additional_objs (list, optional) – Additional linkable object files that should be supplied to the linker/archiver if dont_link is False. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method and the associated linker/archiver’s call method if dont_link is False.

Returns

Output to stdout from the command execution if skip_flags is

True, produced file otherwise.

Return type

str

combine_with_linker = None
compile_only_flag = '-c'
default_archiver = None
default_archiver_flags = None
default_linker = None
default_linker_flags = None
flag_options = {'definitions': '-D%s', 'include_dirs': '-I%s'}
classmethod get_flags(dont_link=None, add_linker_switch=False, libtype=None, logging_level=None, **kwargs)[source]

Get a list of compiler flags.

Parameters
  • dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable or library. Defaults to True if cls.no_separate_linking is True or libtype is ‘object’ and False otherwise.

  • add_linker_switch (bool, optional) – If True, the linker_switch flag will be added to the flags even if dont_link is True as long as the flag is not None. Defaults to False.

  • libtype (str, optional) – Library type that should be created by the linker/archiver. Defaults to None.

  • logging_level (int, optional) – Logging level that should be passed as a definition to the C compiler. Defaults to None and will be ignored.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method and get_linker_flags if dont_link is False.

Returns

Compiler flags.

Return type

list

Raises

ValueError – If dont_link is False and cls.combine_with_linker is False.

classmethod get_library_tool(libtype=None, **kwargs)[source]

Determine the tool that should be used based on the provided arguments.

Parameters
  • libtype (str, optional) – Library type that should be created by the linker/archiver. If ‘static’, the archiver is returned. If ‘shared’ or any other value, the linker is returned. Defaults to None.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Linker/archiver that should be used.

Return type

CompilationToolBase

classmethod get_output_file(src, dont_link=False, working_dir=None, libtype=None, no_src_ext=False, no_tool_suffix=False, suffix='', **kwargs)[source]

Determine the appropriate output file that will result when compiling a given source file.

Parameters
  • src (str) – Source file being compiled that name base will be taken from.

  • dont_link (bool, optional) – If True, the result assumes that the source is just compiled and not linked. If False, the result will be the final result after linking. Defaults to None and will be set to True if libtype is ‘object’ and False otherwise.

  • working_dir (str, optional) – Working directory where output file should be located. Defaults to None and is ignored.

  • libtype (str, optional) – Library type that should be created by the linker/archiver. Defaults to None.

  • no_src_ext (bool, optional) – If True, the source extension will not be added to the object file name. Defaults to False. Ignored if dont_link is False.

  • no_tool_suffix (bool, optional) – If True, the tool suffix will not be added to the object file name. Defaults to False.

  • suffix (str, optional) – Suffix that should be added to the output file (before the extension). Defaults to “”.

  • **kwargs – Additional keyword arguments are ignored unless dont_link is False; then they are passed to the linker’s get_output_file method.

Returns

Full path to file that will be produced.

Return type

str

is_linker = True
classmethod linker()[source]

Get the associated linker class.

Returns

Linker class associated with this compiler.

Return type

CompilationToolBase

linker_attributes = {}
linker_base_classes = None
linker_switch = None
no_separate_linking = False
object_ext = '.o'
search_path_env = ['include']
tooltype = 'compiler'
class yggdrasil.drivers.CompiledModelDriver.DummyLinkerBase(**kwargs)[source]

Bases: LinkerBase

Base class for a dummy linker in the case that the linking step cannot be split into a separate call.

classmethod call(*args, **kwargs)[source]

Raises an error to ward off calling the dummy linker.

classmethod get_flags(**kwargs)[source]

Raises an error to ward off getting flags for the dummy linker.

is_dummy = True
toolname = 'dummy'
class yggdrasil.drivers.CompiledModelDriver.GCCLinker(**kwargs)

Bases: LinkerBase

aliases = ['gnu-cc']
compatible_toolsets = ['gnu']
default_executable = 'gcc'
default_executable_env = 'LD'
default_flags = []
default_flags_env = 'LDFLAGS'
flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}, 'library_rpath': '-Wl,-rpath'}
is_gnu = True
languages = ['c']
library_ext = '.so'
platforms = ['MacOS', 'Linux', 'Windows']
search_path_envvar = ['LIBRARY_PATH', 'LD_LIBRARY_PATH']
search_path_flags = ['-Xlinker', '--verbose']
search_regex = ['SEARCH_DIR\\("=([^"]+)"\\);']
toolname = 'gcc'
toolset = 'gnu'
class yggdrasil.drivers.CompiledModelDriver.GFortranLinker(**kwargs)

Bases: LinkerBase

aliases = []
compatible_toolsets = ['gnu']
default_executable = 'gfortran'
default_executable_env = 'FC'
default_flags = []
default_flags_env = 'LFLAGS'
is_gnu = True
languages = ['fortran']
library_ext = '.so'
platforms = ['MacOS', 'Linux', 'Windows']
search_path_envvar = ['LIBRARY_PATH', 'LD_LIBRARY_PATH']
toolname = 'gfortran'
toolset = 'gnu'
class yggdrasil.drivers.CompiledModelDriver.GPPLinker(**kwargs)

Bases: LinkerBase

aliases = ['gnu-c++']
compatible_toolsets = ['gnu']
default_executable = 'g++'
default_executable_env = 'LD'
default_flags = []
default_flags_env = 'LDFLAGS'
flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}, 'library_rpath': '-Wl,-rpath'}
is_gnu = True
languages = ['c++']
library_ext = '.so'
platforms = ['MacOS', 'Linux', 'Windows']
search_path_envvar = ['LIBRARY_PATH', 'LD_LIBRARY_PATH']
search_path_flags = ['-Xlinker', '--verbose']
search_regex = ['SEARCH_DIR\\("=([^"]+)"\\);']
toolname = 'g++'
toolset = 'gnu'
class yggdrasil.drivers.CompiledModelDriver.LinkerBase(**kwargs)[source]

Bases: CompilationToolBase

Base class for linkers.

shared_library_flag

Flag that should be prepended to the linker tool arguments to indicate that a shared/dynamic library should be produced instead of an executable.

Type

str

library_name_key

Option key indicating the name of a library that should be linked against.

Type

str

library_directory_key

Option key indicating a directory that should be included in the linker search path for libraries.

Type

str

library_prefix

Prefix that should be added to library paths.

Type

str

library_ext

Extension that should be used for shared libraries.

Type

str

executable_ext

Extension that should be used for executables.

Type

str

output_first_library

If True, the output key (and its value) are put in front of the other flags when building a library. A value of None causes the output_first attribute to be used (unless explicitly set in the method call).

Type

bool

all_library_ext = ['.so', '.a']
static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

executable_ext = '.out'
classmethod extract_kwargs(kwargs, compiler=None, add_kws_link=[], add_kws_both=[])[source]

Extract linker kwargs, leaving behind just compiler kwargs.

Parameters
  • kwargs (dict) – Keyword arguments passed to the compiler that should be sorted into kwargs used by either the compiler or linker or both. Keywords that are not used by the compiler will be removed from this dictionary.

  • compiler (CompilerBase, optional) – Compiler tool that linker kwargs are being extracted in order to call. Defaults to None and is ignored.

  • add_kws_link (list, optional) – Addition keywords that should be added to the list of those reserved for the linker. Defaults to [].

  • add_kws_both (list, optional) – Additional keywords that should be added to the list of those that are valid for both the linker and compiler. Defaults to [].

Returns

Keyword arguments that should be passed to the linker.

Return type

dict

flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}}
classmethod get_flags(build_library=False, skip_library_libs=False, use_library_path=False, **kwargs)[source]

Get a list of linker flags.

Parameters
  • libraries (list, optional) – Full paths to libraries that should be linked against. Defaults to an empty list.

  • library_dirs (list, optional) – Directories that should be searched for libraries. Defaults to an empty list.

  • library_libs (list, optional) – Names of libraries that should be linked against. Defaults to an empty list.

  • library_libs_nonstd (list, optional) – Names of libraries w/ non-standard naming conventions that should be linked against. Defaults to an empty list.

  • library_flags (list, optional) – Existing list that library flags should be appended to instead of the returned flags if skip_library_libs is True. Defaults to [].

  • build_library (bool, optional) – If True, a shared library is built. If False, an executable is created. Defaults to False.

  • skip_library_libs (bool, optional) – If True, the library_libs will not be added to the returned flags. Instead, any additional required library flags will be appended to the provided library_flags list which should then be added to the compilation command by the user in the appropriate location. Defaults to False.

  • use_library_path (bool, optional) – If True, the included libraries will be added to the output list as complete paths rather than as separate flags for library and library search directory. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Linker flags.

Return type

list

classmethod get_output_file(obj, build_library=False, working_dir=None, suffix='', no_tool_suffix=False, **kwargs)[source]

Determine the appropriate output file that will result when linking a given object file.

Parameters
  • obj (str) – Object file being linked that name base will be taken from.

  • build_library (bool, optional) – If True, a shared library path is returned. If False, an executable file name is returned. Defaults to False.

  • working_dir (str, optional) – Working directory where output file should be located. Defaults to None and is ignored.

  • suffix (str, optional) – Suffix that should be added to the output file (before the extension). Defaults to “”.

  • no_tool_suffix (bool, optional) – If True, the tool suffix will not be added to the object file name. Defaults to False.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Full path to file that will be produced.

Return type

str

classmethod is_standard_libname(libname)[source]

Determine if the provided file name conforms to the standards expected by this linker.

Parameters

libname (str) – Library file name to check.

Returns

True if the name conforms, False otherwise.

Return type

bool

classmethod libpath2libname(libpath)[source]

Determine the library name from the library path.

Parameters

libpath (str) – Full or partial path to library.

Returns

Library name.

Return type

str

library_ext = None
library_prefix = 'lib'
output_first_library = None
search_path_env = ['lib']
shared_library_flag = '-shared'
tooltype = 'linker'
class yggdrasil.drivers.CompiledModelDriver.LockedFile(fname, context, when_to_lock='init')[source]

Bases: object

Class for locking files during compilation.

property message

Message form.

Type

str

class yggdrasil.drivers.CompiledModelDriver.MSVCLinker(**kwargs)

Bases: LinkerBase

aliases = []
compatible_toolsets = ['msvc']
default_executable = 'LINK'
default_executable_env = 'LINK'
default_flags = []
default_flags_env = 'LDFLAGS'
flag_options = {'library_dirs': '/LIBPATH:%s', 'library_libs': '', 'library_libs_nonstd': ''}
is_gnu = False
languages = ['c']
library_ext = '.so'
output_first = True
output_first_library = False
output_key = '/OUT:%s'
platforms = ['Windows']
search_path_envvar = ['LIB']
search_path_flags = ['-Xlinker', '--verbose']
search_regex = ['SEARCH_DIR\\("=([^"]+)"\\);']
shared_library_flag = '/DLL'
toolname = 'LINK'
toolset = 'msvc'
class yggdrasil.drivers.CompiledModelDriver.MakeLinker(**kwargs)

Bases: DummyLinkerBase

aliases = []
default_executable = 'make'
default_executable_env = None
default_flags = []
executable_ext = ''
is_gnu = False
languages = ['make']
library_ext = '.so'
platforms = ['MacOS', 'Linux']
tool_suffix_format = ''
toolname = 'make'
toolset = None
class yggdrasil.drivers.CompiledModelDriver.NMakeLinker(**kwargs)

Bases: DummyLinkerBase

aliases = []
default_executable = 'nmake'
default_executable_env = None
default_flags = []
executable_ext = ''
is_gnu = False
languages = ['make']
library_ext = '.so'
platforms = ['Windows']
tool_suffix_format = ''
toolname = 'nmake'
toolset = None
yggdrasil.drivers.CompiledModelDriver.find_compilation_tool(tooltype, language, allow_failure=False, dont_check_installation=False)[source]

Return the prioritized class for a compilation tool of a certain type that can handle the specified language.

Parameters
  • tooltype (str) – Type of tool. Valid values include ‘compiler’, ‘linker’, and ‘archiver’.

  • allow_failure (bool, optional) – If True and a tool cannot be located, None will be returned. Otherwise, an error will be raised if a tool cannot be located. Defaults to False.

  • dont_check_installation (bool, optional) – If True, the first tool in the registry will be returned even if it is not installed. Defaults to False.

Returns

Name of the determined tool type.

Return type

str

Raises

RuntimeError – If a tool cannot be located for the specified language on the current platform and allow_failure is False.

yggdrasil.drivers.CompiledModelDriver.get_compatible_tool(tool, tooltype, language, default=False)[source]

Get a compatible compilation tool that can be used in conjunction with the one provided based on the registry of compilation toolsets.

Parameters
  • tool (CompilationToolBase, str) – Compilation tool or name of a compilation tool to get compatible counterpart to.

  • tooltype (str) – Type of compilation tool that should be returned.

  • language (str) – Language that compilation tool should handle.

  • default (CompilationToolBase, optional) – Default tool that should be returned if not compatible tool can be located. Defaults to False and an error will be raised if a tool cannot be located.

Returns

Compatible compilation tool class.

Return type

CompilationToolBase

yggdrasil.drivers.CompiledModelDriver.get_compilation_tool(tooltype, name, default=False)[source]

Return the class providing information about a compilation tool.

Parameters
  • tooltype (str) – Type of tool. Valid values include ‘compiler’, ‘linker’, and ‘archiver’.

  • name (str) – Name or path to the desired compilation tool.

  • default (object, optional) – Value that should be returned if a tool cannot be located. If False, an error will be raised. Defaults to False.

Returns

Class providing access to the specified tool.

Return type

CompilationToolBase

Raises

ValueError – If a tool with the provided name cannot be located.

yggdrasil.drivers.CompiledModelDriver.get_compilation_tool_registry(tooltype, init_languages=None)[source]

Return the registry containing compilation tools of the specified type.

Parameters
  • tooltype (str) – Type of tool. Valid values include ‘compiler’, ‘linker’, and ‘archiver’.

  • init_languages (list, optional) – List of languages that should be imported prior to returning the registry, thereby populating the compilation tools for that language. Defaults to None and is ignored.

Returns

Registry for specified type.

Return type

collections.OrderedDict

Raises

ValueError – If tooltype is not a valid value (i.e. ‘compiler’, ‘linker’, or ‘archiver’).

yggdrasil.drivers.ConnectionDriver module

Module for funneling messages from one comm to another.

class yggdrasil.drivers.ConnectionDriver.ConnectionDriver(*args, **kwargs)[source]

Bases: Driver

Class that continuously passes messages from one comm to another.

Parameters
  • name (str) – Name that should be used to set names of input/output comms.

  • inputs (list, optional) – One or more dictionaries containing keyword arguments for constructing input communicators. Defaults to an empty dictionary if not provided.

  • outputs (list, optional) – One or more dictionaries containing keyword arguments for constructing output communicators. Defaults to an empty dictionary if not provided.

  • input_pattern (str, optional) –

    The communication pattern that should be used to handle incoming messages when there is more than one input communicators present. Defaults to ‘cycle’. Options include:

    ’cycle’: Receive from the next available input communicator. ‘gather’: Receive lists of messages with one element from each

    communicator where a message is only returned when there is a message from each.

  • output_pattern (str, optional) –

    The communication pattern that should be used to handling outgoing messages when there is more than one output communicator present. Defaults to ‘broadcast’. Options include:

    ’cycle’: Rotate through output comms, sending one message to

    each.

    ’broadcast’: Send the same message to each comm. ‘scatter’: Send part of message (must be a list) to each comm.

  • translator (str, func, optional) – Function or string specifying function that should be used to translate messages from the input communicator before passing them to the output communicator. If a string, the format should be “<package.module>:<function>” so that <function> can be imported from <package>. Defaults to None and messages are passed directly. This can also be a list of functions/strings that will be called on the messages in the order they are provided.

  • timeout_send_1st (float, optional) – Time in seconds that should be waited before giving up on the first send. Defaults to self.timeout.

  • single_use (bool, optional) – If True, the driver will be stopped after one loop. Defaults to False.

  • onexit (str, optional) – Class method that should be called when a model that the connection interacts with exits, but before the connection driver is shut down. Defaults to None.

  • **kwargs – Additonal keyword arguments are passed to the parent class.

icomm_kws

Keyword arguments for the input communicator.

Type

dict

ocomm_kws

Keyword arguments for the output communicator.

Type

dict

icomm

Input communicator.

Type

CommBase

ocomm

Output communicator.

Type

CommBase

nrecv

Number of messages received.

Type

int

nproc

Number of messages processed.

Type

int

nsent

Number of messages sent.

Type

int

state

Descriptor of last action taken.

Type

str

translator

Function that will be used to translate messages from the input communicator before passing them to the output communicator.

Type

func

timeout_send_1st

Time in seconds that should be waited before giving up on the first send.

Type

float

single_use

If True, the driver will be stopped after one loop.

Type

bool

onexit

Class method that should be called when the corresponding model exits, but before the driver is shut down.

Type

str

after_loop()[source]

Actions to perform after sending messages.

after_loop_process()[source]

Actions to preform after loop for process.

before_loop()[source]

Actions to perform prior to sending messages.

property can_run_remotely

True if process should be run remotely.

Type

bool

cleanup()[source]

Ensure that the communicators are closed.

close_comm()[source]

Close the communicators.

property close_state

State of the connection at close.

Type

str

confirm_input(timeout=None)[source]

Confirm receipt of messages from input comm.

confirm_output(timeout=None)[source]

Confirm receipt of messages from output comm.

do_terminate()[source]

Stop the driver by closing the communicators.

drain_input(timeout=None)[source]

Drain messages from input comm.

drain_output(timeout=None, dont_confirm_eof=False)[source]

Drain messages from output comm.

get_flag_attr(attr)[source]

Return the flag attribute.

graceful_stop(timeout=None, **kwargs)[source]

Stop the driver, first waiting for the input comm to be empty.

Parameters
  • timeout (float, optional) – Max time that should be waited. Defaults to None and is set to attribute timeout.

  • **kwargs – Additional keyword arguments are passed to the parent class’s graceful_stop method.

property is_comm_closed

Returns True if both communicators are closed.

Type

bool

property is_comm_open

Returns True if both communicators are open.

Type

bool

property is_valid

Returns True if the connection is open and the parent class is valid.

Type

bool

property model_env

Mapping between model name and opposite comm environment variables that need to be provided to the model.

Type

dict

property n_msg

Number of messages waiting in input communicator.

Type

int

property nproc

Number of messages processed.

Type

int

property nrecv

Number of messages received.

Type

int

property nsent

Number of messages sent.

Type

int

on_eof(msg)[source]

Actions to take when EOF received.

Parameters

msg (CommMessage) – Message object that provided the EOF.

Returns

Value that should be returned by recv_message on EOF.

Return type

CommMessage, bool

on_message(msg)[source]

Process a message.

Parameters

msg (bytes, str) – Message to be processed.

Returns

Processed message.

Return type

bytes, str

on_model_exit(direction, name, errors=False)[source]

Drain input and then close it.

on_model_exit_remote(direction, name, errors=False)[source]

Drain input and then close it (on the remote process).

Parameters
  • direction (str) – Direction of model.

  • name (str) – Name of model exiting.

  • errors (list, optional) – Errors generated by the model. Defaults to False.

Returns

True if all of the input/output models have signed

off; False otherwise.

Return type

bool

open_comm()[source]

Open the communicators.

printStatus(beg_msg='', end_msg='', verbose=False, return_str=False)[source]

Print information on the status of the ConnectionDriver.

Parameters
  • beg_msg (str, optional) – Additional message to print at beginning.

  • end_msg (str, optional) – Additional message to print at end.

  • verbose (bool, optional) – If True, the status of individual comms will be displayed. Defaults to False.

  • return_str (bool, optional) – If True, the message string is returned. Defaults to False.

recv_message(**kwargs)[source]

Get a new message to send.

Parameters

**kwargs – Additional keyword arguments are passed to the appropriate recv method.

Returns

False if no more messages, message otherwise.

Return type

CommMessage, bool

remove_model(direction, name)[source]

Remove a model from the list of models.

Parameters
  • direction (str) – Direction of model.

  • name (str) – Name of model exiting.

Returns

True if all of the input/output models have signed

off; False otherwise.

Return type

bool

run_loop()[source]

Run the driver. Continue looping over messages until there are not any left or the communication channel is closed.

send_eof(**kwargs)[source]

Send EOF message.

Returns

Success or failure of send.

Return type

bool

send_message(msg, **kwargs)[source]

Send a single message.

Parameters
  • msg (CommMessage) – Message being sent.

  • *kwargs – Keyword arguments are passed to the output comm send method.

Returns

Success or failure of send.

Return type

bool

set_close_state(state)[source]

Set the close state if its not already set.

set_flag_attr(attr, value=True)[source]

Set a flag.

start()[source]

Open connection before running.

property state

Current state of the connection.

Type

str

update_serializer(msg)[source]

Update the serializer for the output comm based on input.

wait_for_route(timeout=None)[source]

Wait until messages have been routed.

class yggdrasil.drivers.ConnectionDriver.RemoteTaskLoop(*args, **kwargs)[source]

Bases: YggTaskLoop

Class to handle running tasks on the connection loop process.

after_loop()[source]

Actions performed after the loop.

close()[source]

Close the queues.

is_open()[source]
run_task_local(task, args, kwargs)[source]

Run task on the current process.

run_task_remote(task, args, kwargs)[source]

Run task on the connection loop process.

target()[source]

Complete all pending tasks.

exception yggdrasil.drivers.ConnectionDriver.TaskThreadError[source]

Bases: RuntimeError

yggdrasil.drivers.ConnectionDriver.run_remotely(method)[source]

Decorator for methods that should be run remotely.

yggdrasil.drivers.DSLModelDriver module

class yggdrasil.drivers.DSLModelDriver.DSLModelDriver(*args, **kwargs)[source]

Bases: InterpretedModelDriver

Class for running domain specific lanugage models.

after_loop()[source]

Actions to perform after run_loop has finished. Mainly checking if there was an error and then handling it.

base_languages = ['python']
executable_type = 'dsl'
full_language = False
function_param = None
is_dsl = True
classmethod is_language_installed()[source]

Determine if this model driver is installed on the current machine.

Returns

Truth of if this model driver can be run on the current

machine.

Return type

bool

classmethod is_library_installed(lib, **kwargs)[source]

Determine if a dependency is installed.

Parameters
  • lib (str) – Name of the library that should be checked.

  • **kwargs – Additional keyword arguments are ignored.

Returns

True if the library is installed, False otherwise.

Return type

bool

classmethod language_version(**kwargs)[source]

Determine the version of this language.

Parameters

**kwargs – Keyword arguments are passed to cls.run_executable.

Returns

Version of compiler/interpreter for this language.

Return type

str

classmethod model_wrapper(*args, **kwargs)[source]

Model wrapper.

property model_wrapper_args

Positional arguments for the model wrapper.

Type

tuple

property model_wrapper_kwargs

Keyword arguments for the model wrapper.

Type

dict

classmethod model_wrapper_no_forward(*args, **kwargs)[source]
queue_close()[source]

Close the queue for messages from the model process.

queue_recv()[source]

Receive a message from the model process.

run_model(return_process=True, **kwargs)[source]

Run the model. Unless overridden, the model will be run using run_executable.

Parameters
  • return_process (bool, optional) – If True, the process running the model is returned. If False, the process will block until the model finishes running. Defaults to True.

  • **kwargs – Keyword arguments are passed to run_executable.

yggdrasil.drivers.Driver module

class yggdrasil.drivers.Driver.Driver(*args, **kwargs)[source]

Bases: YggTaskLoop

Base class for all drivers.

Parameters
  • name (str) – Driver name.

  • yml (dict, optional) – Dictionary of yaml specification options for this driver. Defaults to empty dict.

  • env (dict, optional) – Dictionary of environment variables that should be set when the driver starts. Defaults to {}.

  • comm_env (dict, optional) – Dictionary of environment variables for paired IO communication drivers. Defaults to {}.

  • namespace (str, optional) – Namespace for set of drivers running together. If not provided, the config option (‘rmq’, ‘namespace’) is used.

  • rank (int, optional) – Rank of the integration. Defaults to None.

name

Driver name.

Type

str

env

Dictionary of environment variables.

Type

dict

comm_env

Dictionary of environment variables for paired IO communication drivers.

Type

dict

yml

Dictionary of yaml specification options for this driver.

Type

dict

namespace

Namespace for set of drivers running together.

Type

str

rank

Rank of the integration.

Type

int

do_terminate()[source]

Actions that should stop the driver.

graceful_stop()[source]

Gracefully stop the driver.

property is_valid

True if the driver is functional.

Type

bool

stop()[source]

Stop the driver.

terminate()[source]

Stop the driver, without attempting to allow it to finish.

yggdrasil.drivers.DummyModelDriver module

class yggdrasil.drivers.DummyModelDriver.DummyModelDriver(*args, **kwargs)[source]

Bases: InterpretedModelDriver

Class that stands-in to act as a model utilizing unmatched input/output channels.

base_languages = ['python']
before_loop()[source]

Actions before loop.

before_start()[source]

Actions to perform before the run starts.

cfg = <yggdrasil.config.YggConfigParser object>
comms_implicit = True
default_interpreter = 'dummy'
default_interpreter_flags = []
executable_type = 'other'
full_language = False
inverse_type_map = None
classmethod is_configured()[source]

Determine if the appropriate configuration has been performed (e.g. installation of supporting libraries etc.)

Returns

True if the language has been configured.

Return type

bool

classmethod is_language_installed()[source]

Determine if this model driver is installed on the current machine.

Returns

Truth of if this model driver can be run on the current

machine.

Return type

bool

language = 'dummy'
language_ext = []
classmethod language_version(**kwargs)[source]

Determine the version of this language.

Parameters

**kwargs – Keyword arguments are passed to cls.run_executable.

Returns

Version of compiler/interpreter for this language.

Return type

str

no_executable = True
run_loop()[source]

Loop to check if model is still running and forward output.

property service_partner

YAML representation of the dummy model that should stand-in for the model client-side.

Type

dict

yggdrasil.drivers.DuplicatedModelDriver module

class yggdrasil.drivers.DuplicatedModelDriver.DuplicatedModelDriver(*args, **kwargs)[source]

Bases: Driver

Base class for Model drivers and for running executable based models.

Parameters
  • name (str) – Unique name used to identify the model. This will be used to report errors associated with the model.

  • *args – Additional arguments are passed to the models in the set.

  • **kwargs – Additional keyword arguments are passed to the models in the set.

Attributes:

Raises

RuntimeError – If both with_strace and with_valgrind are True.

after_loop(*args, **kwargs)[source]

Actions to perform after run_loop has finished.

cleanup(*args, **kwargs)[source]

Actions to perform to clean up the thread after it has stopped.

property errors

Errors returned by model copies.

Type

list

classmethod get_base_name(name)[source]

Get the name of the base model.

Parameters

name (str) – Model name.

Returns

Base model name.

Return type

str

classmethod get_yaml_copies(yml)[source]

Get a list of yamls for creating duplicate models for the model described by the provided yaml.

Parameters

yml (dict) – Input parameters for creating a model driver.

Returns

Copies of input parameters for creating duplicate models.

Return type

list

graceful_stop(*args, **kwargs)[source]

Gracefully stop the driver.

property io_errors

Errors produced by input/output drivers to this model.

Type

list

name_format = '%s_copy%d'
printStatus(*args, **kwargs)[source]

Print the class status.

run_loop()[source]

Loop to check if model is still running and forward output.

start(*args, **kwargs)[source]

Start thread/process and print info.

stop(*args, **kwargs)[source]

Stop the driver.

terminate(*args, **kwargs)[source]

Set the terminate event and wait for the thread/process to stop.

yggdrasil.drivers.ExecutableModelDriver module

class yggdrasil.drivers.ExecutableModelDriver.ExecutableModelDriver(*args, **kwargs)[source]

Bases: ModelDriver

Class for running executable based models.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

cfg = <yggdrasil.config.YggConfigParser object>
comms_implicit = True
classmethod executable_command(args, unused_kwargs=None, **kwargs)[source]

Compose a command for running a program using the exectuable for this language (compiler/interpreter) with the provided arguments.

Parameters
  • args (list) – The program that returned command should run and any arguments that should be provided to it.

  • unused_kwargs (dict, optional) – Existing dictionary that unused keyword arguments should be added to. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Arguments composing the command required to run the program

from the command line using the executable for this language.

Return type

list

executable_type = 'other'
full_language = False
inverse_type_map = None
classmethod is_configured()[source]

Determine if the appropriate configuration has been performed (e.g. installation of supporting libraries etc.)

Returns

True if the language has been configured.

Return type

bool

classmethod is_language_installed()[source]

Determine if the interpreter/compiler for the associated programming language is installed.

Returns

True if the language interpreter/compiler is installed.

Return type

bool

language = 'executable'
classmethod language_version(version_flags=None, **kwargs)[source]

Determine the version of this language.

Parameters

**kwargs – Keyword arguments are passed to cls.run_executable.

Returns

Version of compiler/interpreter for this language.

Return type

str

no_executable = True

yggdrasil.drivers.FileInputDriver module

class yggdrasil.drivers.FileInputDriver.FileInputDriver(*args, **kwargs)[source]

Bases: InputDriver

Class that sends messages read from a file.

Parameters
  • name (str) – Name of the queue that messages should be sent to.

  • args (str) – Path to the file that messages should be read from.

  • **kwargs – Additional keyword arguments are passed to the parent class.

yggdrasil.drivers.FileOutputDriver module

class yggdrasil.drivers.FileOutputDriver.FileOutputDriver(*args, **kwargs)[source]

Bases: OutputDriver

Class to handle output of received messages to a file.

Parameters
  • name (str) – Name of the output queue to receive messages from.

  • args (str) – Path to the file that messages should be written to.

  • **kwargs – Additional keyword arguments are passed to the parent class.

yggdrasil.drivers.FortranModelDriver module

class yggdrasil.drivers.FortranModelDriver.FortranCompilerBase(**kwargs)[source]

Bases: CompilerBase

Base class for Fortran compilers.

classmethod append_product(products, src, new, new_dir=None, dont_append_src=False)[source]

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters
  • products (list) – List of of existing products that new product should be appended to.

  • src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).

  • new (str) – New product that should be appended to the list.

  • new_dir (str, optional) – Directory that should be used as base when adding files listed in cls.product_files. Defaults to os.path.dirname(new).

  • dont_append_src (bool, optional) – If True and src is in the list of products, it will be removed. Defaults to False.

default_archiver = None
default_executable = None
default_executable_env = 'FC'
default_flags = ['-g', '-Wall', '-cpp', '-pedantic-errors', '-ffree-line-length-0']
default_flags_env = 'FFLAGS'
default_linker = None
classmethod get_flags(**kwargs)[source]

Get a list of flags for the tool.

Parameters

**kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Flags for the tool.

Return type

list

languages = ['fortran']
linker_attributes = {'default_flags_env': 'LFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH']}
product_exts = ['mod']
search_path_envvar = []
class yggdrasil.drivers.FortranModelDriver.FortranModelDriver(*args, **kwargs)[source]

Bases: CompiledModelDriver

Class for running Fortran models.

Parameters
  • standard (str, optional) – Fortran standard that should be used. Defaults to ‘f2003’.

  • **kwargs – Additional keyword arguments are passed to parent class.

classmethod allows_realloc(var, from_native_type=False)[source]

Determine if a variable allows the receive call to perform realloc.

Parameters
  • var (dict) – Dictionary of variable properties.

  • from_native_type (bool, optional) – If True, the reallocability of the variable will be determined from the native type. Defaults to False.

Returns

True if the variable allows realloc, False otherwise.

Return type

bool

base_languages = ['c']
static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

cfg = <yggdrasil.config.YggConfigParser object>
compile_model(**kwargs)[source]

Compile model executable(s).

Parameters
  • **kwargs – Keyword arguments are passed to the parent class’s

  • method.

Returns

Compiled model file path.

Return type

str

default_archiver = 'ar'
default_archiver_flags = None
default_compiler = 'gfortran'
default_compiler_flags = None
default_linker = 'gfortran'
default_linker_flags = None
classmethod escape_quotes(x)[source]

Escape quotes in a string.

Parameters

x (str) – String to escape quotes in.

Returns

x with escaped quotes.

Return type

str

external_libraries = {'stdc++': {'include': 'stdlib.h', 'language': 'c', 'libtype': 'shared'}}
classmethod format_function_param_index(extra=None, **kwargs)[source]

Return the formatted version of the index key.

Parameters
  • extra (dict, optional) – Variable dictionary specifying the variable name, datatype, etc. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns

Formatted string.

Return type

str

classmethod format_function_param_len(extra=None, **kwargs)[source]

Return the formatted version of the len key.

Parameters
  • extra (dict, optional) – Variable dictionary specifying the variable name, datatype, etc. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns

Formatted string.

Return type

str

function_param = {'and': '.and.', 'assign': '{name} = {value}', 'block_end': 'END', 'break': 'EXIT', 'comment': '!', 'continuation_after': '     &', 'continuation_before': '&', 'copy_any': '{name} = copy_generic({value})', 'copy_array': '{name} = yggarr(copy_generic(ygggeneric({value})))', 'copy_class': '{name} = copy_python({value})', 'copy_function': '{name} = yggpyfunc(copy_python(yggpython({value})))', 'copy_generic': '{name} = copy_generic({value})', 'copy_instance': '{name} = yggpyinst(copy_generic(ygggeneric({value})))', 'copy_obj': '{name} = copy_obj({value})', 'copy_object': '{name} = yggmap(copy_generic(ygggeneric({value})))', 'copy_ply': '{name} = copy_ply({value})', 'copy_schema': '{name} = yggschema(copy_generic(ygggeneric({value})))', 'declare': '{type_name} :: {variable}', 'definition_regex': '\\s*(?P<type>(?:(?: )|(?:.))+)\\s*::\\s*(?P<name>.+)(?:\\s*=\\s*(?P<value>.+))?\\n', 'error': 'write(*, \'("{error_msg}")\')\nstop 1', 'exec_begin': 'PROGRAM main\n   use iso_c_binding\n   use fygg', 'exec_end': '   stop\nEND PROGRAM main', 'false': '.false.', 'flag_cond': '{flag_var}', 'for_begin': 'DO {iter_var} = {iter_begin}, {iter_end}', 'for_end': 'END DO', 'fprintf': 'write(*, \'("{message}")\') {variables}', 'free': 'DEALLOCATE({variable})', 'free_any': 'call free_generic({variable})', 'free_array': 'call free_generic(ygggeneric({variable}))', 'free_class': 'call free_python({variable})', 'free_function': 'call free_python(yggpython({variable}))', 'free_instance': 'call free_generic(ygggeneric({variable}))', 'free_obj': 'call free_obj({variable})', 'free_object': 'call free_generic(ygggeneric({variable}))', 'free_ply': 'call free_ply({variable})', 'free_schema': 'call free_generic(ygggeneric({variable}))', 'function_call_noout': 'call {function_name}({input_var})', 'function_def_begin': 'FUNCTION {function_name}({input_var}) result({output_var})', 'function_def_end': 'END FUNCTION {function_name}', 'function_def_regex': '(?P<procedure_type>(?i:(?:subroutine)|(?:function)))\\s+{function_name}\\s*\\((?P<inputs>(?:[^\\(]*?))\\)\\s*(?:result\\s*\\((?P<flag_var>.+)\\))?\\s*\\n(?P<preamble>(?:[^:]*?\\n)*?)(?P<definitions>(?:(?:(?: )|(?:.))+\\s*::\\s*(?:.+)\\n)+)(?P<body>(?:.*?\\n?)*?)(?i:end\\s+(?P=procedure_type))\\s+{function_name}', 'if_begin': 'IF ({cond}) THEN', 'if_elif': 'ELSE IF ({cond}) THEN', 'if_else': 'ELSE', 'if_end': 'END IF', 'import': '#include "{filename}"', 'import_nofile': 'use {function}', 'indent': '   ', 'index': '{variable}({index})', 'init_any': 'init_generic()', 'init_array': 'yggarr(init_generic())', 'init_class': 'init_python()', 'init_function': 'yggpyfunc(init_python())', 'init_instance': 'yggpyinst(init_generic())', 'init_obj': 'init_obj()', 'init_object': 'yggmap(init_generic())', 'init_ply': 'init_ply()', 'init_schema': 'yggschema(init_generic())', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': '{channel} = ygg_input_type("{channel_name}", {channel_type})', 'inputs_def_regex': '\\s*(?P<name>.+?)\\s*(?:,|$)(?:\\n)?', 'interface': 'use fygg', 'len': 'size({variable},1)', 'not': '.not.', 'not_flag_cond': '.not.{flag_var}', 'null': 'c_null_ptr', 'or': '.or.', 'output': '{channel} = ygg_output_type("{channel_name}", {channel_type})', 'outputs_def_regex': '\\s*(?P<name>.+?)\\s*(?:,|$)(?:\\n)?', 'print': 'write(*, \'("{message}")\')', 'print_any': 'call display_generic({object})', 'print_array': 'call display_generic(ygggeneric({object}))', 'print_class': 'call display_python({object})', 'print_function': 'call display_python(yggpython({object}))', 'print_generic': 'write(*, *) {object}', 'print_instance': 'call display_generic(ygggeneric({object}))', 'print_null': 'call display_null({object})', 'print_obj': 'call display_obj({object})', 'print_object': 'call display_generic(ygggeneric({object}))', 'print_ply': 'call display_ply({object})', 'print_schema': 'call display_generic(ygggeneric({object}))', 'quote': "'", 'recv_function': 'ygg_recv_var_realloc', 'recv_heap': 'ygg_recv_var_realloc', 'recv_stack': 'ygg_recv_var', 'send_function': 'ygg_send_var', 'subroutine_def_begin': 'SUBROUTINE {function_name}({input_var}) ', 'subroutine_def_end': 'END SUBROUTINE {function_name}', 'true': '.true.', 'type_regex': '(type\\()?(?P<type>[^,\\(]+)(?(1)(?:\\)))(?:\\s*\\(\\s*(?:kind\\s*=\\s*(?:(?P<precision>\\d*)|(?P<precision_var>.+?)))?\\s*,?\\s*(?:len\\s*=\\s*(?:(?P<length>(?:\\d+))|(?P<length_var>.+?)))?\\))?(?:\\s*,\\s*dimension\\((?:(?P<shape>(?:\\d)+(?:,\\s*(?:\\d)+)*?)|(?P<shape_var>.+?(?:,\\s*.+)*?))\\))?(?:\\s*,\\s*(?P<pointer>pointer))?(?:\\s*,\\s*(?P<target>target))?(?:\\s*,\\s*(?P<allocatable>allocatable))?(?:\\s*,\\s*(?P<parameter>parameter))?(?:\\s*,\\s*intent\\((?P<intent>.*?)\\))?', 'while_begin': 'DO WHILE ({cond})', 'while_end': 'END DO'}
classmethod get_internal_suffix(commtype=None)[source]

Determine the suffix that should be used for internal libraries.

Parameters

commtype (str, optional) – If provided, this is the communication type that should be used for the model. If None, the default comm is used.

Returns

Suffix that should be added to internal libraries to

differentiate between different dependencies.

Return type

str

classmethod get_inverse_type_map()[source]

Get the inverse type map.

Returns

Mapping from native type to JSON type.

Return type

dict

classmethod get_json_type(native_type)[source]

Get the JSON type from the native language type.

Parameters

native_type (str) – The native language type.

Returns

The JSON type.

Return type

str, dict

classmethod get_native_type(**kwargs)[source]

Get the native type.

Parameters
  • type (str, optional) – Name of yggdrasil extended JSON type or JSONSchema dictionary defining a datatype.

  • **kwargs – Additional keyword arguments may be used in determining the precise declaration that should be used.

Returns

The native type.

Return type

str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class.

Returns

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type

dict

global_scope_macro = '#define WITH_GLOBAL_SCOPE(COMM) call set_global_comm(); COMM; call unset_global_comm()'
include_channel_obj = True
interface_inside_exec = True
interface_library = 'fygg'
interface_map = {'call': 'flag = ygg_rpc_call({channel_obj}, [yggarg({outputs})], [yggarg({inputs})])', 'client': 'ygg_rpc_client_type("{channel_name}", {datatype_out}, {datatype_in})', 'import': 'use fygg', 'input': 'ygg_input_type("{channel_name}", {datatype})', 'output': 'ygg_output_type("{channel_name}", {datatype})', 'recv': 'flag = ygg_recv_var({channel_obj}, [yggarg({inputs})])', 'send': 'flag = ygg_send_var({channel_obj}, [yggarg({outputs})])', 'server': 'ygg_rpc_server_type("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'yggTimesync("{channel_name}", "{time_units}")'}
internal_libraries = {'c_wrappers': {'external_dependencies': [('c', 'rapidjson'), ('c', 'python'), ('c', 'numpy')], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/fortran'], 'internal_dependencies': [('c', 'ygg')], 'language': 'c', 'libtype': 'object', 'source': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/fortran/c_wrappers.c'}, 'fygg': {'external_dependencies': [('c', 'rapidjson'), ('c', 'python'), ('c', 'numpy'), 'stdc++'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'internal_dependencies': [('c', 'regex'), ('c', 'datatypes'), ('c', 'python_wrapper'), ('c', 'ygg'), 'c_wrappers'], 'libtype': 'static', 'source': '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/fortran/YggInterface.f90'}}
inverse_type_map = None
is_typed = True
language = 'fortran'
language_ext = ['.f90', '.f77', '.f', '.h']
locked_buildfile = 'fygg.mod'
max_line_width = 72
outputs_in_inputs = True
classmethod parse_function_definition(model_file, model_function, **kwargs)[source]

Get information about the inputs & outputs to a model from its defintition if possible.

Parameters
  • model_file (str) – Full path to the file containing the model function’s declaration.

  • model_function (str) – Name of the model function.

  • **kwargs – Additional keyword arugments are passed to the parent class’s method.

Returns

Parameters extracted from the function definitions.

Return type

dict

classmethod prepare_variables(vars_list, for_yggdrasil=False, **kwargs)[source]

Concatenate a set of input variables such that it can be passed as a single string to the function_call parameter.

Parameters
  • vars_list (list) – List of variable dictionaries containing info (e.g. names) that should be used to prepare a string representing input/output to/from a function call.

  • for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Concatentated variables list.

Return type

str

classmethod set_env_class(**kwargs)[source]

Set environment variables that are instance independent.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.

Returns

Environment variables for the model process.

Return type

dict

classmethod split_line(line, length=None, force_split=False)[source]

Split a line as close to (or before) a given character as possible.

Parameters
  • line (str) – Line to split.

  • length (int, optional) – Maximum length of split lines. Defaults to cls.max_line_width if not provided.

  • force_split (bool, optional) – If True, force a split to occur at the specified length. Defauts to False.

Returns

Set of lines resulting from spliting the provided line.

Return type

list

supported_comm_options = {'ipc': {'platforms': ['MacOS', 'Linux']}, 'zmq': {'libraries': [('c', 'zmq'), ('c', 'czmq')]}}
type_map = {'1darray': '*', '1darray_pointer': '{type}{precision}_1d', 'any': 'ygggeneric', 'array': 'yggarr', 'boolean': 'logical(kind = X)', 'bytes': 'character(len = X)', 'class': 'yggpython', 'comm': 'yggcomm', 'complex': 'complex(kind = X)', 'dtype': 'yggdtype', 'flag': 'logical', 'float': 'real(kind = X)', 'function': 'yggpyfunc', 'instance': 'yggpyinst', 'int': 'integer(kind = X)', 'integer': 'integer', 'ndarray': '*', 'ndarray_pointer': '{type}{precision}_{ndim}d', 'null': 'yggnull', 'obj': 'yggobj', 'object': 'yggmap', 'ply': 'yggply', 'schema': 'yggschema', 'string': 'character(len = X)', 'uint': 'ygguintX', 'unicode': "character(kind = selected_char_kind('ISO_10646'), len = X)"}
types_in_funcdef = False
classmethod update_io_from_function(model_file, model_function, **kwargs)[source]

Update inputs/outputs from the function definition.

Parameters
  • model_file (str) – Full path to the file containing the model function’s declaration.

  • model_function (str) – Name of the model function.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Flag variable used by the model. If None, the

model does not use a flag variable.

Return type

dict, None

classmethod write_declaration(var, **kwargs)[source]

Return the lines required to declare a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines declaring the variable.

Return type

list

classmethod write_executable(lines, **kwargs)[source]

Return the lines required to complete a program that will run the provided lines.

Parameters
  • lines (list) – Lines of code to be wrapped as an executable.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines of code wrapping the provided lines with the

necessary code to run it as an executable (e.g. C/C++’s main).

Return type

lines

classmethod write_executable_import(module=False, **kwargs)[source]

Add import statements to executable lines.

Parameters
  • module (str, optional) – If provided, the include statement importing code will be wrapped in a module of the provided name. Defaults to False and the include will not be wrapped.

  • **kwargs – Keyword arguments for import statement.

Returns

Lines required to complete the import.

Return type

list

classmethod write_function_def(function_name, **kwargs)[source]

Write a function definition.

Parameters
  • function_name (str) – Name fo the function being defined.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines completing the function call.

Return type

list

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]

Get the lines necessary to initialize an array for iteration output.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being initialized.

  • value (str, optional) – Value that should be assigned to the variable.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines initializing the variable.

Return type

list

classmethod write_model_recv(channel, recv_var, **kwargs)[source]

Write a model receive call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being received from was stored in.

  • recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a receive call in this language.

Return type

list

classmethod write_model_send(channel, send_var, **kwargs)[source]

Write a model send call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being sent to was stored in.

  • send_var (dict, list) – Information on one or more variables containing information that will be sent.

  • flag_var (str, optional) – Name of flag variable that the flag should be stored in. Defaults to ‘flag’,

  • allow_failure (bool, optional) – If True, the returned lines will call a break if the flag is False. Otherwise, the returned lines will issue an error. Defaults to False.

Returns

Lines required to carry out a send call in this language.

Return type

list

classmethod write_print_var(var, **kwargs)[source]

Get the lines necessary to print a variable in this language.

Parameters
  • var (dict) – Variable information.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines printing the specified variable.

Return type

list

classmethod write_type_decl(name, datatype, **kwargs)[source]

Get lines declaring the datatype within the language.

Parameters
  • name (str) – Name of variable that should be declared.

  • datatype (dict) – Type definition.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to define a type declaration.

Return type

list

classmethod write_type_def(name, datatype, **kwargs)[source]

Get lines declaring the data type within the language.

Parameters
  • name (str) – Name of variable that definition should be stored in.

  • datatype (dict) – Type definition.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to define a type definition.

Return type

list

zero_based = False
class yggdrasil.drivers.FortranModelDriver.GFortranCompiler(**kwargs)[source]

Bases: FortranCompilerBase

Interface class for gfortran compiler/linker.

combine_with_linker = True
compatible_toolsets = ['gnu', 'llvm']
default_archiver = 'ar'
default_executable = 'gfortran'
default_flags = ['-g', '-Wall', '-cpp', '-pedantic-errors', '-ffree-line-length-0', '-x', 'f95-cpp-input']
default_linker = 'gfortran'
flag_options = {'definitions': '-D%s', 'include_dirs': '-I%s', 'module-dir': '-J%s', 'module-search-path': '-I%s', 'standard': '-std=%s'}
is_gnu = True
platforms = ['MacOS', 'Linux', 'Windows']
toolname = 'gfortran'
toolset = 'gnu'

yggdrasil.drivers.InputDriver module

class yggdrasil.drivers.InputDriver.InputDriver(*args, **kwargs)[source]

Bases: ConnectionDriver

Driver for receiving input from another model’s comm via a local comm.

Parameters
  • name (str) – The name of the local message comm that the driver should connect to.

  • args (str) – The name of the other message comm that the driver should connect to.

  • **kwargs – Additional keyword arguments are passed to parent class.

yggdrasil.drivers.InterpretedModelDriver module

class yggdrasil.drivers.InterpretedModelDriver.InterpretedModelDriver(*args, **kwargs)[source]

Bases: ModelDriver

Base class for models written in interpreted languages.

Parameters
  • name (str) – Driver name.

  • args (str or list) – Argument(s) for running the model on the command line. This driver will check to see if there is an interpreter included in the args provided. If not, one will be added unless skip_interpreter is True.

  • interpreter (str, optional) – Name or path of interpreter executable that should be used to run the model. If not provided, the interpreter will be determined based on configuration options for the language (if present) and the default_interpreter class attribute.

  • interpreter_flags (list, optional) – Flags that should be passed to the interpreter when running the model. If not provided, the flags are determined based on configuration options for the language (if present) and the default_interpreter_flags class attribute.

  • skip_interpreter (bool, optional) – If True, no interpreter will be added to the arguments. This should only be used for subclasses that will not be invoking the model via the command line. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class.

Class Attributes:
default_interpreter (str): Name of interpreter that will be used if not

set explicitly by instance or config file. Defaults to the language name if not set.

default_interpreter_flags (list): Flags that will be passed to the

interpreter when running the model by default if not set explicitly by instance or config file.

interpreter

Name or path to the interpreter that will be used.

Type

str

interpreter_flags

Flags that will be passed to the interpreter when running a model.

Type

list

path_env_variable

Name of the environment variable containing path information for the interpreter for this language.

Type

str

paths_to_add

Paths that should be added to the path_env_variable for this language on the process the model is run in.

Type

list

comm_atexit

Function taking a comm instance as input that performs any necessary operations during exit. If None, no additional actions are taken.

Type

function

comm_linger

If True, interface comms will linger during close. This should only be required if the language will disreguard Python threads at exit (e.g. when using a Matlab engine).

Type

bool

decode_format (function

Function decoding format string created in this language. If None, no additional actions are taken.

recv_converters

Mapping between the names of message types (e.g. ‘array’, ‘pandas’) and functions that should be used to prepare such objects for return when they are received.

Type

dict

send_converters

Mapping between the names of message types (e.g. ‘array’, ‘pandas’) and functions that should be used to prepare such objects for sending.

Type

dict

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

comm_atexit = None
comm_linger = False
decode_format = None
default_interpreter = None
default_interpreter_flags = []
classmethod executable_command(args, exec_type='interpreter', unused_kwargs=None, skip_interpreter_flags=False, interpreter=None, interpreter_flags=None, **kwargs)[source]

Compose a command for running a program in this language with the provied arguments. If not already present, the interpreter command and interpreter flags are prepended to the provided arguments.

Parameters
  • args (list) – The program that returned command should run and any arguments that should be provided to it.

  • exec_type (str, optional) – Type of executable command that will be returned. If ‘interpreter’, a command using the interpreter is returned and if ‘direct’, the raw args being provided are returned. Defaults to ‘interpreter’.

  • skip_interpreter_flags (bool, optional) – If True, interpreter flags will not be added to the command after the interpreter. Defaults to False. Interpreter flags will not be added, reguardless of this keyword, if the first element of args is already an interpreter.

  • unused_kwargs (dict, optional) – Existing dictionary that unused keyword arguments should be added to. Defaults to {}.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Arguments composing the command required to run the program

from the command line using the interpreter for this language.

Return type

list

Raises

ValueError – If exec_type is not ‘interpreter’ or ‘direct’.

executable_type = 'interpreter'
classmethod get_interpreter()[source]

Command required to run a model written in this language from the command line.

Returns

Name of (or path to) interpreter executable.

Return type

str

classmethod get_interpreter_flags()[source]

Get the flags that should be passed to the interpreter when using it to run a model on the command line.

Returns

Flags that should be passed to the interpreter on the command

line.

Return type

list

classmethod is_interpreter(cmd)[source]

Determine if a command line argument is an interpreter.

Parameters

cmd (str) – Command that should be checked.

Returns

True if the command is an interpreter, False otherwise.

Return type

bool

classmethod language2python(pyobj)[source]

Prepare an object from the target language for receipt in Python.

Parameters

pyobj (object) – Python object transformed from the target language.

Returns

Python object in a form that conforms with the

expected Python type.

Return type

object

classmethod language_executable()[source]

Command required to compile/run a model written in this language from the command line.

Returns

Name of (or path to) compiler/interpreter executable required

to run the compiler/interpreter from the command line.

Return type

str

parse_arguments(*args, **kwargs)[source]

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters
  • *args – Arguments are passed to the parent class’s method.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

path_env_variable = None
paths_to_add = ['/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/Python', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/R', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/MATLAB', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/SBML', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/LPy', '/opt/hostedtoolcache/Python/3.8.13/x64/lib/python3.8/site-packages/yggdrasil/languages/dummy']
classmethod python2language(pyobj)[source]

Prepare a python object for transformation in target language.

Parameters

pyobj (object) – Python object.

Returns

Python object in a form that is friendly to the

target language.

Return type

object

recv_converters = {}
run_model(*args, **kwargs)[source]

Run the model. Unless overridden, the model will be run using run_executable.

Parameters
  • *args – Arguments are passed to the parent class’s method.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

send_converters = {}
set_env(**kwargs)[source]

Get environment variables that should be set for the model process.

Returns

Environment variables for the model process.

Return type

dict

yggdrasil.drivers.LPyModelDriver module

class yggdrasil.drivers.LPyModelDriver.LPyModelDriver(*args,