yggdrasil.languages.Python package¶

yggdrasil.languages.Python.YggInterface.YggArrayOutput(name, **kwargs)[source]¶

Get class for handling table-like formatted output as arrays.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiArrayInput(name, **kwargs)[source]¶

Get class for handling table-like formatted input as arrays.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggAsciiTableInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiArrayOutput(name, fmt=None, **kwargs)[source]¶

Get class for handling table-like formatted output as arrays.

Parameters:

name (str) – The name of the message queue where output should be sent.
fmt (str) – A C style format string specifying how each ‘row’ of output should be formated. This should include the newline character.
**kwargs – Additional keyword arguments are passed to YggAsciiTableOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiFileInput(name, **kwargs)[source]¶

Get class for generic ASCII input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiFileOutput(name, **kwargs)[source]¶

Get class for generic ASCII output.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiTableInput(name, as_array=False, **kwargs)[source]¶

Get class for handling table-like formatted input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
as_array (bool, optional) – If True, recv returns the entire table array and can only be called once. If False, recv returns row entries. Default to False.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggAsciiTableOutput(name, fmt=None, as_array=False, **kwargs)[source]¶

Get class for handling table-like formatted output.

Parameters:

name (str) – The name of the message queue where output should be sent.
fmt (str) – A C style format string specifying how each ‘row’ of output should be formated. This should include the newline character.
as_array (bool, optional) – If True, send expects and entire array. If False, send expects the entries for one table row. Defaults to False.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggInit(_type, args=None, kwargs=None)[source]¶

Short interface to identify functions called by another language.

Parameters:

_type (str) – Name of class that should be returned.
args (list, optional) – Additional arguments that should be passed to the class initializer.
kwargs (dict, optional) – Additional keyword arguments that should be passed to the class initializer.

Returns:

An instance of the requested class.

Return type:

obj

yggdrasil.languages.Python.YggInterface.YggInput(name, format_str=None, **kwargs)[source]¶

Get class for handling input from a message queue.

Parameters:

name (str) – The name of the message queue. Combined with the suffix ‘_IN’, it should match an environment variable containing a message queue key.
format_str (str, optional) – C style format string that should be used to deserialize messages that are receieved into a list of python objects. Defaults to None and raw string messages are returned.
**kwargs – Additional keyword arguments are passed to the underlying comm.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggObjInput(name, **kwargs)[source]¶

Get class for handling Obj input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggObjOutput(name, **kwargs)[source]¶

Get class for handling Obj output.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggOutput(name, format_str=None, **kwargs)[source]¶

Get class for handling output to a message queue.

Parameters:

name (str) – The name of the message queue. Combined with the suffix ‘_OUT’, it should match an environment variable containing a message queue key.
format_str (str, optional) – C style format string that should be used to create a message from a list of python ojbects. Defaults to None and raw string messages are sent.
**kwargs – Additional keyword arguments are passed to the underlying comm.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPandasInput(name, **kwargs)[source]¶

Get class for handling Pandas input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPandasOutput(name, **kwargs)[source]¶

Get class for handling Pandas output.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPickleInput(name, **kwargs)[source]¶

Get class for handling pickled input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPickleOutput(name, **kwargs)[source]¶

Get class for handling pickled output.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPlyInput(name, **kwargs)[source]¶

Get class for handling Ply input.

Parameters:

name (str) – The name of the input message queue that input should be received from.
**kwargs – Additional keyword arguments are passed to YggInput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggPlyOutput(name, **kwargs)[source]¶

Get class for handling Ply output.

Parameters:

name (str) – The name of the message queue where output should be sent.
**kwargs – Additional keyword arguments are passed to YggOutput.

Returns:

Communication object.

Return type:

yggdrasil.languages.Python.YggInterface.YggRpcClient(name, outfmt=None, infmt=None, **kwargs)[source]¶

Get class for handling requests and response to an RPC Server from a client.

Parameters:

name (str) – The name of the server queues.
outfmt (str, optional) – Format string used to format variables in a message sent to the request queue. Defautls to ‘%s’.
infmt (str, optional) – Format string used to recover variables from messages received from the response queue. Defautls to ‘%s’.
**kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns:

class:.ClientComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggRpcServer(name, infmt=None, outfmt=None, **kwargs)[source]¶

Get class for handling requests and response for an RPC Server.

Parameters:

name (str) – The name of the server queues.
infmt (str, optional) – Format string used to recover variables from messages received from the request queue. Defaults to ‘%s’.
outfmt (str, optional) – Format string used to format variables in a message sent to the response queue. Defautls to ‘%s’.
**kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns:

class:.ServerComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggTimesync(name='timesync', **kwargs)[source]¶

Get class for handling timestep synchronization requests.

Parameters:

name (str, optional) – The name of the server queues. Defaults to ‘timesync’.
**kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns:

class:.ClientComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggTimesyncServer(name='timesync', **kwargs)[source]¶

Get class for handling timestep synchronization requests on the driver side.

Parameters:

name (str, optional) – The name of the server queues. Defaults to ‘timesync’.
**kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns:

class:.ServerComm: Communication object.

yggdrasil.languages.Python.YggInterface.bufMsgSize()[source]¶: int: Buffer size for average message.

yggdrasil.languages.Python.YggInterface.eof_msg()[source]¶: str: Message signalling end of file.

yggdrasil.languages.Python.YggInterface.maxMsgSize()[source]¶: int: The maximum message size.

Module contents¶

yggdrasil.serialize package¶

Submodules¶

yggdrasil.serialize.AsciiMapSerialize module¶

class yggdrasil.serialize.AsciiMapSerialize.AsciiMapSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for serializing/deserializing name/value mapping.

Parameters:: delimiter (str, optional) – Delimiter that should be used to separate name/value pairs in the map. Defaults to t.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = False¶

default_datatype = {'type': 'object'}¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (bytes) – Message to be deserialized.
Returns:: Deserialized Python dictionary.
Return type:: dict

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (dict) – Python dictionary to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

yggdrasil.serialize.AsciiTableSerialize module¶

class yggdrasil.serialize.AsciiTableSerialize.AsciiTableSerialize(*args, **kwargs)[source]¶

Bases: DefaultSerialize

Class for serialize table output into bytes messages comprising a formatted ASCII table.

Parameters:

format_str (str, optional) – If provided, this string will be used to format messages from a list of arguments and parse messages to get a list of arguments in C printf/scanf style. Defaults to None and messages are assumed to already be bytes.
field_names (list, optional) – The names of fields in the format string. If not provided, names are set based on the order of the fields in the format string.
field_units (list, optional) – The units of fields in the format string. If not provided, all fields are assumed to be dimensionless.
as_array (bool, optional) – If True, each of the arguments being serialized/deserialized will be arrays that are converted to/from bytes in column major (‘F’) order. Otherwise, each argument should be a scalar. Defaults to False.
delimiter (str, optional) – Character(s) that should be used to separate columns. Defaults to ‘t’.
use_astropy (bool, optional) – If True, the astropy package will be used to serialize/deserialize table. Defaults to False.
**kwargs – Additional keyword args are processed as part of the type definition.

format_str¶

Format string used to format/parse bytes messages from/to a list of arguments in C printf/scanf style.

Type:: str

field_names¶

The names of fields in the format string.

Type:: list

field_units¶

The units of fields in the format string.

Type:: list

as_array¶

True or False depending if serialized/deserialized python objects will be arrays or scalars.

Type:: bool

delimiter¶

Character(s) that should be used to separate columns.

Type:: str

use_astropy¶

If True, the astropy package will be used to serialize/deserialize table.

Type:: bool

default_datatype = {'type': 'array'}¶

default_read_meth = 'readline'¶

deserialize_file_header(fd)[source]¶

Deserialize the header information from the file and update the serializer.

Parameters:: fd (file) – File containing header.

file_extensions = ['.txt']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:: **kwargs – Keyword arguments are passed to the parent class’s method.
Returns:: Dictionary of variables to use for testing.
Return type:: dict

has_header = True¶

normalize(args)[source]¶

Normalize a message to conform to the expected datatype.

Parameters:: args (object) – Message arguments.
Returns:: Normalized message.
Return type:: object

property read_meth¶

Method that should be used to read data for deserialization.

Type:: str

serialize_file_header()[source]¶

Return the serialized header information that should be prepended to files serialized using this class.

Returns:: Header string that should be written to the file.
Return type:: bytes

update_field_names()[source]¶: list: Names for each field in the data type.

update_field_units()[source]¶: list: Units for each field in the data type.

update_format_str()[source]¶: Update the format string based on the type definition.

update_serializer(*args, **kwargs)[source]¶

Update serializer with provided information.

Parameters:

skip_type (bool, optional) – If True, everything is updated except the data type. Defaults to False.
**kwargs – Additional keyword arguments are processed as part of they type definition and are parsed for old-style keywords.

Raises:

RuntimeError – If there are keywords that are not valid datatype keywords (currect or old-style).

yggdrasil.serialize.CABOSerialize module¶

class yggdrasil.serialize.CABOSerialize.CABOSerialize(*args, **kwargs)[source]¶

Bases: AsciiMapSerialize

Class for serializing/deserializing CABO parameter files.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = True¶

default_datatype = {'type': 'object'}¶

file_extensions = ['.cab']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (bytes) – Message to be deserialized.
Returns:: Deserialized Python dictionary.
Return type:: dict

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (dict) – Python dictionary to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

classmethod parse_units(x)[source]¶

Parse units.

Parameters:: x (str) – Unit string.
Returns:: Propertly formatted units.
Return type:: str

yggdrasil.serialize.DefaultSerialize module¶

class yggdrasil.serialize.DefaultSerialize.DefaultSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Default class for serializing/deserializing a python object into/from a bytes message.

Parameters:: **kwargs – Additional keyword args are passed to the parent class.

classmethod concatenate(objects, as_array=False, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

classmethod dict2object(obj, as_array=False, field_names=None, **kwargs)[source]¶

Conver a dictionary to a message object.

Parameters:

obj (dict) – Dictionary to convert to serializable object.
as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.
field_names (list, optional) – The field names associated with a table-like data type. Defaults to None. This keyword must be provided if as_array is True.
**kwargs – Additional keyword arguments are ignored.

Returns:

Serializable object.

Return type:

file_extensions = ['.ygg']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶: Method to return a dictionary of testing options for this class.

classmethod object2array(obj, as_array=False, field_names=None, **kwargs)[source]¶

Convert a message object into an array.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in an array form.
as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.
field_names (list, optional) – The field names associated with a table-like data type. Defaults to None. This keyword must be provided if as_array is True.
**kwargs – Additional keyword arguments are ignored.

Returns:

Array version of the provided object.

Return type:

np.array

classmethod object2dict(obj, as_array=False, field_names=None, **kwargs)[source]¶

Convert a message object into a dictionary.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.
as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.
field_names (list, optional) – The field names associated with a table-like data type. Defaults to None. This keyword must be provided if as_array is True.
**kwargs – Additional keyword arguments are ignored.

Returns:

Dictionary version of the provided object.

Return type:

yggdrasil.serialize.DirectSerialize module¶

class yggdrasil.serialize.DirectSerialize.DirectSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for directly serializing bytes.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

default_datatype = {'type': 'bytes'}¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for comms tested with the: provided content.
empty (object): Object produced from deserializing an empty: message.

objects (list): List of objects to be serialized/deserialized. extra_kwargs (dict): Extra keyword arguments not used to

construct type definition.

typedef (dict): Type definition resulting from the supplied: kwargs.
dtype (np.dtype): Numpy data types that is consistent with the: determined type definition.

Return type:

yggdrasil.serialize.FunctionalSerialize module¶

class yggdrasil.serialize.FunctionalSerialize.FunctionalSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for serializing/deserializing a Python object into/from a bytes message using defined functions.

Parameters:

encoded_datatype (schema, optional) – JSON schema describing the type that serialized objects should conform to. Defaults to the class attribute default_encoded_datatype. If either func_serialize or func_deserialize are not provided, this needs to be specified in order to serialize non-bytes objects.
func_serialize (func, optional) – Callable object that takes Python objects as input and returns a representation that conforms to encoded_datatype. Defaults to None and the default serialization for encoded_datatype will be used.
func_deserialize (func, optional) – Callable object that takes objects of a type that conforms to encoded_datatype and returns a deserialized Python object. Defaults to None and the default deserialization for encoded_datatype will be used.
**kwargs – Additional keyword args are passed to the parent class’s constructor.

func_deserialize = None¶

func_serialize = None¶

property serializer_info¶

Serializer info.

Type:: dict

yggdrasil.serialize.JSONSerialize module¶

class yggdrasil.serialize.JSONSerialize.JSONSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for serializing a python object into a bytes message using JSON.

Parameters:

indent (str, int, optional) – String or number of spaces that should be used to indent each level within the seiralized structure. Defaults to ‘t’.
sort_keys (bool, optional) – If True, the serialization of dictionaries will be in key sorted order. Defaults to True.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = False¶

default_datatype = {'type': 'object'}¶

file_extensions = ['.json']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (str, bytes) – Message to be deserialized.
Returns:: Deserialized Python object.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (obj) – Python object to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

yggdrasil.serialize.JSONSerialize.decode_json(msg, **kwargs)[source]¶

Decode a Python object from a JSON serialization.

Parameters:

msg (str) – JSON serialization to decode.
**kwargs – Additional keyword arguments are passed to rapidjson.loads.

Returns:

Deserialized Python object.

Return type:

yggdrasil.serialize.JSONSerialize.encode_json(obj, fd=None, indent=None, sort_keys=True, **kwargs)[source]¶

Encode a Python object in JSON format.

Parameters:

obj (object) – Python object to encode.
fd (file, optional) – File descriptor for file that encoded object should be written to. Defaults to None and string is returned.
indent (int, str, optional) – Indentation for new lines in encoded string. Defaults to None.
sort_keys (bool, optional) – If True, the keys will be output in sorted order. Defaults to True.
**kwargs – Additional keyword arguments are passed to rapidjson.dumps.

Returns:

Encoded object.

Return type:

str, bytes

yggdrasil.serialize.JSONSerialize.indent_char2int(indent)[source]¶

Convert a character indent into a number of spaces that should be used. Tabs are set to be equivalent to 4 spaces.

Parameters:: indent (str) – String indent.
Returns:: Number of whitespaces that is equivalent to the provided string.
Return type:: int

yggdrasil.serialize.MatSerialize module¶

class yggdrasil.serialize.MatSerialize.MatSerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for serializing a python object into a bytes message using the Matlab .mat format.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = False¶

default_datatype = {'type': 'object'}¶

file_extensions = ['.mat']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (str, bytes) – Message to be deserialized.
Returns:: Deserialized Python object.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (obj) – Python object to be serialized.
Returns:: Serialized message.
Return type:: bytes, str
Raises:: TypeError – If args is not a dictionary.

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

yggdrasil.serialize.ObjSerialize module¶

class yggdrasil.serialize.ObjSerialize.ObjDict[source]¶

Bases: GeometryBase, ObjWavefront

Enhanced dictionary class for storing Obj information.

apply_scalar_map(*args, **kwargs)[source]¶

Set the color of faces in a 3D object based on a scalar map. This creates a copy unless no_copy is True.

Parameters:

scalar_arr (arr) – Scalar values that should be mapped to colors for each face.
color_map (str, optional) – The name of the color map that should be used. Defaults to ‘plasma’.
vmin (float, optional) – Value that should map to the minimum of the colormap. Defaults to min(scalar_arr).
vmax (float, optional) – Value that should map to the maximum of the colormap. Defaults to max(scalar_arr).
scaling (str, optional) – Scaling that should be used to map the scalar array onto the colormap. Defaults to ‘linear’.
scale_by_area (bool, optional) – If True, the elements of the scalar array will be multiplied by the area of the corresponding face. If True, vmin and vmax should be in terms of the scaled array. Defaults to False.
no_copy (bool, optional) – If True, the returned object will not be a copy. Defaults to False.

Returns:

Obj with updated vertex colors.

Return type:

classmethod from_shape(shape, d, conversion=1.0)[source]¶

Create a ply dictionary from a PlantGL shape and descritizer.

Parameters:

scene (openalea.plantgl.scene) – Scene that should be descritized.
d (openalea.plantgl.descritizer) – Descritizer.
conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

to_geom_args(conversion=1.0, name=None)[source]¶

Get arguments for creating a PlantGL geometry.

Parameters:

conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.
name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns:

Class, arguments and keyword arguments for PlantGL geometry.

Return type:

tuple

class yggdrasil.serialize.ObjSerialize.ObjSerialize(*args, **kwargs)[source]¶

Bases: PlySerialize

Class for serializing/deserializing .obj file formats. Reader adapted from https://www.pygame.org/wiki/OBJFileLoader.

default_datatype = {'type': 'obj'}¶

file_extensions = ['.obj']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (bytes) – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

normalize(args)[source]¶

Normalize a message to conform to the expected datatype.

Parameters:: args (object) – Message arguments.
Returns:: Normalized message.
Return type:: object

yggdrasil.serialize.PandasSerialize module¶

class yggdrasil.serialize.PandasSerialize.PandasSerialize(*args, **kwargs)[source]¶

Bases: AsciiTableSerialize

Class for serializing/deserializing Pandas data frames.

Parameters:

no_header (bool, optional) – If True, headers will not be read or serialized from/to tables. Defaults to False.
str_as_bytes (bool, optional) – If True, strings in columns are read as bytes. Defaults to False.

classmethod apply_field_names(frame, field_names=None)[source]¶

Apply field names as columns to a frame, first checking for a mapping. If there is a direct mapping, the columns are reordered to match the order of the field names. If there is not an overlap in the field names and columns, a one-to-one mapping is assumed, but a warning is issued. If there is a partial overlap, an error is raised.

Parameters:

frame (pandas.DataFrame) – Frame to apply field names to as columns.
field_names (list, optional) – New field names that should be applied. If not provided, the original frame will be returned unaltered.

Returns:

Frame with updated field names.

Return type:

pandas.DataFrame

Raises:

RuntimeError – If there is a partial overlap between the field names and columns.

as_array = True¶

cformat2nptype(*args, **kwargs)[source]¶

Method to convert c format string to numpy data type.

Parameters:

*args – Arguments are passed to serialize.cformat2nptype.
**kwargs – Keyword arguments are passed to serialize.cformat2nptype.

Returns:

Corresponding numpy data type.

Return type:

np.dtype

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = False¶

consolidate_array(out)[source]¶

Consolidate message into a structure numpy array if possible.

Parameters:: out (list, tuple, np.ndarray) – Object to consolidate into a structured numpy array.
Returns:: Structured numpy array containing consolidated message.
Return type:: np.ndarray
Raises:: ValueError – If the array cannot be consolidated.

default_read_meth = 'read'¶

deserialize_file_header(fd)[source]¶

Deserialize the header information from the file and update the serializer.

Parameters:: fd (file) – File containing header.

classmethod dict2object(obj, field_names=None, **kwargs)[source]¶

Conver a dictionary to a message object.

Parameters:

obj (dict) – Dictionary to convert to serializable object.
**kwargs – Additional keyword arguments are ignored.

Returns:

Serializable object.

Return type:

disable_file_header()[source]¶: Set serializer attributes to disable file headers from being included in the serializations.

property empty_msg¶

Object indicating empty message.

Type:: obj

enable_file_header()[source]¶: Set serializer attributes to enable file headers to be included in the serializations.

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (str, bytes) – Message to be deserialized.
Returns:: Deserialized Python object.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (obj) – Python object to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

get_field_names(*args, **kwargs)[source]¶

Get the field names for an array of fields.

Parameters:

*args – Arguments are passed to the parent class’s method.
**kwargs – Keyword arguments are passed to the parent class’s method.

Returns:

Names for each field in the data type.

Return type:

classmethod get_testing_options(not_as_frames=False, no_names=False, no_header=False, **kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

not_as_frames (bool, optional) – If True, the returned example includes data that is not in a pandas data frame. Defaults to False.
no_names (bool, optional) – If True, an example is returned where the names are not provided to the deserializer. Defaults to False.
no_header (bool, optional) – If True, an example is returned
False. (where a header is not included. Defaults to) –

Returns:

Dictionary of variables to use for testing.

Return type:

initialize_from_message(msg, **kwargs)[source]¶

Initialize the serializer based on recieved message.

Parameters:

msg (object) – Message that serializer should be initialized from.
**kwargs – Additional keyword arguments are treated as metadata that may contain additional information for initializing the serializer.

normalize(args)[source]¶

Normalize a message to conform to the expected datatype.

Parameters:: args (object) – Message arguments.
Returns:: Normalized message.
Return type:: object

classmethod normalize_bytes2unicode(args)[source]¶

Convert columns that are bytes to unicode.

Parameters:: args (pd.DataFrame) – Pandas dataframe to convert columns for.
Returns:: Version of args without bytes in columns.
Return type:: pd.DataFrame

classmethod normalize_unicode2bytes(args)[source]¶

Convert columns that are unicode to bytes.

Parameters:: args (pd.DataFrame) – Pandas dataframe to convert columns for.
Returns:: Version of args without unicode in columns.
Return type:: pd.DataFrame

classmethod object2array(obj, **kwargs)[source]¶

Convert a message object into an array.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in an array form.
**kwargs – Additional keyword arguments are ignored.

Returns:

Array version of the provided object.

Return type:

np.array

classmethod object2dict(obj, **kwargs)[source]¶

Convert a message object into a dictionary.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.
**kwargs – Additional keyword arguments are ignored.

Returns:

Dictionary version of the provided object.

Return type:

property send_converter¶

serialize_file_header()[source]¶

Return the serialized header information that should be prepended to files serialized using this class.

Returns:: Header string that should be written to the file.
Return type:: bytes

yggdrasil.serialize.PickleSerialize module¶

class yggdrasil.serialize.PickleSerialize.PickleSerialize(*args, **kwargs)[source]¶

Bases: DefaultSerialize

Class for serializing a python object into a bytes message by pickling.

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

file_extensions = ['.pkl']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (str, bytes) – Message to be deserialized.
Returns:: Deserialized Python object.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (obj) – Python object to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_first_frame(msg)[source]¶

Extract one frame from the provided message that may contain one or more frames.

Parameters:

msg (bytes) – Message containing one or more frames.

Returns:

Portion of message containing the first frame. If no frames: are found, an empty string will be returned.

Return type:

bytes

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

is_framed = True¶

yggdrasil.serialize.PlySerialize module¶

class yggdrasil.serialize.PlySerialize.GeometryBase[source]¶

Bases: object

Base class for extening rapidjson geometry classes.

apply_scalar_map(scalar_arr, color_map=None, vmin=None, vmax=None, scaling='linear', scale_by_area=False, no_copy=False, _as_obj=False)[source]¶

Set the color of faces in a 3D object based on a scalar map. This creates a copy unless no_copy is True.

Parameters:

scalar_arr (arr) – Scalar values that should be mapped to colors for each face.
color_map (str, optional) – The name of the color map that should be used. Defaults to ‘plasma’.
vmin (float, optional) – Value that should map to the minimum of the colormap. Defaults to min(scalar_arr).
vmax (float, optional) – Value that should map to the maximum of the colormap. Defaults to max(scalar_arr).
scaling (str, optional) – Scaling that should be used to map the scalar array onto the colormap. Defaults to ‘linear’.
scale_by_area (bool, optional) – If True, the elements of the scalar array will be multiplied by the area of the corresponding face. If True, vmin and vmax should be in terms of the scaled array. Defaults to False.
no_copy (bool, optional) – If True, the returned object will not be a copy. Defaults to False.

Returns:

Geometry with updated vertex colors.

Return type:

classmethod from_scene(scene, d=None, conversion=1.0)[source]¶

Create a geometry dictionary from a PlantGL scene and descritizer.

Parameters:

scene (openalea.plantgl.scene) – Scene that should be descritized.
d (openalea.plantgl.descritizer, optional) – Descritizer. Defaults to openalea.plantgl.all.Tesselator.
conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

classmethod from_shape(shape, d, conversion=1.0, _as_obj=False)[source]¶

Create a geometry dictionary from a PlantGL shape and descritizer.

Parameters:

scene (openalea.plantgl.scene) – Scene that should be descritized.
d (openalea.plantgl.descritizer) – Descritizer.
conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

to_geom_args(conversion=1.0, name=None, _as_obj=False)[source]¶

Get arguments for creating a PlantGL geometry.

Parameters:

conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.
name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns:

Class, arguments and keyword arguments for PlantGL geometry.

Return type:

tuple

to_scene(conversion=1.0, name=None)[source]¶

Create a PlantGL scene from a geometry dictionary.

Parameters:

conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.
name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns:

class yggdrasil.serialize.PlySerialize.PlyDict[source]¶

Bases: GeometryBase, Ply

Enhanced dictionary class for storing Ply information.

class yggdrasil.serialize.PlySerialize.PlySerialize(*args, **kwargs)[source]¶

Bases: SerializeBase

Class for serializing/deserializing .ply file formats.

Parameters:

write_header (bool, optional) – If True, headers will be added to serialized output. Defaults to True.
newline (str, optional) – String that should be used for new lines. Defaults to ‘n’.
prune_duplicates (bool, optional) – If True, serialized meshes in array format will be pruned of duplicates when being normalized into a Ply object. If False, duplicates will not be pruned. Defaults to True.

write_header¶

If True, headers will be added to serialized output.

Type:: bool

newline¶

String that should be used for new lines.

Type:: str

default_rgb¶

Default color in RGB that should be used for missing colors.

Type:: list

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = False¶

default_datatype = {'type': 'ply'}¶

file_extensions = ['.ply']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (bytes) – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

property initialized¶

True if the serializer has been initialized.

Type:: bool

classmethod is_mesh(args)[source]¶

Check if an object is a 3D mesh with x, y, z for vertices in each face in rows.

Parameters:: args (object) – Object to check.
Returns:: True if object is a mesh, false otherwise.
Return type:: bool

normalize(args)[source]¶

Normalize a message to conform to the expected datatype.

Parameters:: args (object) – Message arguments.
Returns:: Normalized message.
Return type:: object

yggdrasil.serialize.SerializeBase module¶

class yggdrasil.serialize.SerializeBase.SerializeBase(*args, **kwargs)[source]¶

Bases: YggClass

Base class for serializing/deserializing a Python object into/from a bytes message.

Parameters:

newline (str, optional) – One or more characters indicating a newline. Defaults to ‘n’.
comment (str, optional) – One or more characters indicating a comment. Defaults to ‘# ‘.
datatype (schema, optional) – JSON schema defining the type of object that the serializer will be used to serialize/deserialize. Defaults to default_datatype.
**kwargs – Additional keyword args are processed as part of the type definition.

initialized¶

True if the serializer has been initialized either by input arguments specifying the type or by infering the type from a processed message.

Type:: bool

Class Attributes:

has_header (bool): True if the serialization has a header when written: to a file.
default_read_meth (str): Default method that data should be read from: a file for deserialization.
is_framed (bool): True if the serialization has a frame allowing: multiple serialized objects to be recovered from a single message.
concats_as_str (bool): True if serialized objects can be concatenated: directly as strings.
encoded_datatype (schema): JSON schema defining the type of object: produced by the class’s func_serialize method. For most classes this will be {‘type’: ‘scalar’, ‘subtype’: ‘string’}, indicating that the method will produce bytes suitable for serialization.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration.

cformat2nptype(*args, **kwargs)[source]¶

Method to convert c format string to numpy data type.

Parameters:

*args – Arguments are passed to serialize.cformat2nptype.
**kwargs – Keyword arguments are passed to serialize.cformat2nptype.

Returns:

Corresponding numpy data type.

Return type:

np.dtype

classmethod concatenate(objects, **kwargs)[source]¶

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters:

objects (list) – Objects to be concatenated.
**kwargs – Additional keyword arguments are ignored.

Returns:

Set of objects that results from concatenating those provided.

Return type:

concats_as_str = True¶

consolidate_array(out)[source]¶

Consolidate message into a structure numpy array if possible.

Parameters:: out (list, tuple, np.ndarray) – Object to consolidate into a structured numpy array.
Returns:: Structured numpy array containing consolidated message.
Return type:: np.ndarray
Raises:: ValueError – If the array cannot be consolidated.

decode(msg, no_data=False, metadata=None)[source]¶

Decode message parts into header and body.

Parameters:

msg (str, bytes) – Message to be decoded.
no_data (bool, optional) – If True, only the metadata is returned. Defaults to False.
metadata (dict, optional) – Metadata that should be used to deserialize the message instead of the current header content. Defaults to None and is not used.

Returns:

Deserialized message and header information.

Return type:

tuple(obj, dict)

Raises:

ValueError – If msg contains a header, but metadata is also provided as an argument.
TypeError – If msg is not bytes.

default_datatype = {'subtype': 'string', 'type': 'scalar'}¶

default_read_meth = 'read'¶

deserialize(msg, **kwargs)[source]¶

Deserialize a message.

Parameters:

msg (str, bytes) – Message to be deserialized.
**kwargs – Additional keyword arguments are passed to the decode method.

Returns:

Deserialized message and header information.

Return type:

tuple(obj, dict)

Raises:

TypeError – If msg is not bytes type (str on Python 2).

deserialize_file_header(fd)[source]¶

Deserialize the header information from the file and update the serializer.

Parameters:: fd (file) – File containing header.

classmethod dict2object(obj, **kwargs)[source]¶

Conver a dictionary to a message object.

Parameters:

obj (dict) – Dictionary to convert to serializable object.
**kwargs – Additional keyword arguments are ignored.

Returns:

Serializable object.

Return type:

disable_file_header()[source]¶: Set serializer attributes to disable file headers from being included in the serializations.

dump(fd, obj, **kwargs)[source]¶

Serialize to a file.

Parameters:

fd (str, file) – Filename or file-like object to load from.
**kwargs – Additional keyword arguments are passed to the created FileComm used for reading.

property empty_msg¶

Object indicating empty message.

Type:: obj

enable_file_header()[source]¶: Set serializer attributes to enable file headers to be included in the serializations.

encode(data, metadata, no_metadata=False, max_header_size=0)[source]¶

Encode the message with metadata in a header.

Parameters:

data (bytes) – Message data serialized into bytes.
metadata (dict) – Metadata that should be included in the message header.
no_metadata (bool, optional) – If True, no metadata will be added to the serialized message. Defaults to False.
max_header_size (int, optional) – Maximum size that header should occupy in order to be sent in a single message. A value of 0 indicates that any size header is valid. Defaults to 0.

Returns:

Encoded message with header.

Return type:

bytes

file_extensions = ['.txt']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg – Message to be deserialized.
Returns:: Deserialized message.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args – List of arguments to be formatted or numpy array to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

get_field_names(as_bytes=False)[source]¶

Get the field names for an array of fields.

Parameters:: as_bytes (bool, optional) – If True, the field names will be returned as bytes. If False the field names will be returned as unicode. Defaults to False.
Returns:: Names for each field in the data type.
Return type:: list

get_field_units(as_bytes=False)[source]¶

Get the field units for an array of fields.

Parameters:: as_bytes (bool, optional) – If True, the field units will be returned as bytes. If False the field units will be returned as unicode. Defaults to False.
Returns:: Units for each field in the data type.
Return type:: list

get_status_message(nindent=0, extra_lines_before=None, extra_lines_after=None)[source]¶

Return lines composing a status message.

Parameters:

nindent (int, optional) – Number of tabs that should be used to indent each line. Defaults to 0.
extra_lines_before (list, optional) – Additional lines that should be added to the beginning of the default print message. Defaults to empty list if not provided.
extra_lines_after (list, optional) – Additional lines that should be added to the end of the default print message. Defaults to empty list if not provided.

Returns:

Lines composing the status message and the: prefix string used for the last message.

Return type:

tuple(list, prefix)

classmethod get_testing_options(table_example=False, array_columns=False, include_oldkws=False, table_string_type='bytes', no_names=False, no_units=False, **kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

table_example (bool, optional) – If True, the returned options will be for an array of elements representing a table-like structure. Defaults to False.
array_columns (bool, optional) – If True, table_example is set to True and the returned options will be for an array data type where each element is an array representing a column Defaults to False.
include_oldkws (bool, optional) – If True, old-style keywords will be added to the returned options. This will only have an effect if table_example is True. Defaults to False.
table_string_type (str, optional) – Type that should be used for the string column in the table. Defaults to ‘bytes’.
no_names (bool, optional) – If True, an example is returned where the names are not provided to the deserializer. Defaults to False.
no_units (bool, optional) – If True, units will not be added to the returned array if table_example is True.

Returns:

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for comms tested with the provided content.
empty (object): Object produced from deserializing an empty message.
objects (list): List of objects to be serialized/deserialized. extra_kwargs (dict): Extra keyword arguments not used to construct type definition.
typedef (dict): Type definition resulting from the supplied kwargs.
dtype (np.dtype): Numpy data types that is consistent with the determined type definition.
contents (bytes): Concatenated serialization that will result from deserializing the serialized objects.
contents_recv (list): List of objects that would be deserialized from contents.

Return type:

has_header = False¶

initialize_from_message(msg, serializer=None, **metadata)[source]¶

Initialize the serializer based on recieved message.

Parameters:

msg (object) – Message that serializer should be initialized from.
**kwargs – Additional keyword arguments are treated as metadata that may contain additional information for initializing the serializer.

initialize_from_metadata(metadata)[source]¶

Initialize a serializer based on received metadata. This method will exit early if the serializer has already been intialized.

Parameters:: metadata (dict) – Header information including type info that should be used to initialize the serializer class.

property initialized¶

True if the serializer has been initialized.

Type:: bool

property input_kwargs¶

Get the input keyword arguments used to create this class.

Type:: dict

is_framed = False¶

load(fd, **kwargs)[source]¶

Deserialize from a file.

Parameters:

fd (str, file) – Filename or file-like object to load from.
**kwargs – Additional keyword arguments are passed to the created FileComm used for reading.

Returns:

The deserialized data object or a list of: deserialized data objects if there is more than one.

Return type:

normalize(args)[source]¶

Normalize a message to conform to the expected datatype.

Parameters:: args (object) – Message arguments.
Returns:: Normalized message.
Return type:: object

property numpy_dtype¶

Corresponding structured data type. Will be None unless the type is an array of 1darrays.

Type:: np.dtype

classmethod object2array(obj, **kwargs)[source]¶

Convert a message object into an array.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in an array form.
**kwargs – Additional keyword arguments are ignored.

Returns:

Array version of the provided object or None if one cannot: be created.

Return type:

np.array

classmethod object2dict(obj, **kwargs)[source]¶

Convert a message object into a dictionary.

Parameters:

obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.
**kwargs – Additional keyword arguments are ignored.

Returns:

Dictionary version of the provided object.

Return type:

parse_header(msg)[source]¶

Extract header info from a message.

Parameters:: msg (str) – Message to extract header from.
Returns:: Message properties.
Return type:: dict

property read_meth¶

Method that should be used to read data for deserialization.

Type:: str

classmethod seri_kws()[source]¶: Get a list of valid keyword arguments.

serialize(args, metadata=None, add_serializer_info=False, no_metadata=False, max_header_size=0)[source]¶

Serialize a message.

Parameters:

args (obj) – List of arguments to be formatted or a ready made message.
metadata (dict, optional) – Keyword arguments that should be added to the header. Defaults to None and no header is added.
add_serializer_info (bool, optional) – If True, serializer information will be added to the metadata. Defaults to False.
no_metadata (bool, optional) – If True, no metadata will be added to the serialized message. Defaults to False.
max_header_size (int, optional) – Maximum size that header should occupy in order to be sent in a single message. A value of 0 indicates that any size header is valid. Defaults to 0.

Returns:

Serialized message.

Return type:

bytes, str

Raises:

TypeError – If returned msg is not bytes type (str on Python 2).

serialize_file_header()[source]¶

Return the serialized header information that should be prepended to files serialized using this class.

Returns:: Header string that should be written to the file.
Return type:: bytes

property serializer_info¶

Serializer info.

Type:: dict

property typedef¶

Alias for datatype.

Type:: dict

update_serializer(skip_type=False, seritype=None, datatype=None, from_message=False, **kwargs)[source]¶

Update serializer with provided information.

Parameters:

skip_type (bool, optional) – If True, everything is updated except the data type. Defaults to False.
**kwargs – Additional keyword arguments are processed as part of they type definition and are parsed for old-style keywords.

Raises:

RuntimeError – If there are keywords that are not valid datatype keywords (currect or old-style).

update_typedef_from_oldstyle(typedef)[source]¶

Update a given typedef using an old, table-style serialization spec. Existing typedef values are not overwritten and warnings are raised if the provided serialization spec is not compatible with the type definition.

Parameters:: typedef (dict) – Type definition to update.
Returns:: Updated typedef.
Return type:: dict

yggdrasil.serialize.YAMLSerialize module¶

class yggdrasil.serialize.YAMLSerialize.YAMLSerialize(*args, **kwargs)[source]¶

Bases: JSONSerialize

Class for serializing a python object into a bytes message using YAML.

Parameters:

indent (str, int, optional) – String or number of spaces that should be used to indent each level within the seiralized structure. Defaults to ‘t’.
encoding (str, optional) – Encoding that should be used to serialize the object. Defaults to ‘utf-8’.
default_flow_style (bool, optional) – If True, nested collections will be serialized in the block style. If False, they will always be serialized in the flow style. See PyYAML Documentation.

file_extensions = ['.yaml', '.yml']¶

func_deserialize(msg)[source]¶

Deserialize a message.

Parameters:: msg (str, bytes) – Message to be deserialized.
Returns:: Deserialized Python object.
Return type:: obj

func_serialize(args)[source]¶

Serialize a message.

Parameters:: args (obj) – Python object to be serialized.
Returns:: Serialized message.
Return type:: bytes, str

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Returns:: Dictionary of variables to use for testing.
Return type:: dict

yggdrasil.serialize.YAMLSerialize.decode_yaml(msg, sorted_dict_type=None, **kwargs)[source]¶

Decode a Python object from a YAML serialization.

Parameters:

msg (str) – YAML serialization to decode.
sorted_dict_type (type, optional) – Class that should be used to contain mapping objects while preserving order. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are passed to yaml.load.

Returns:

Deserialized Python object.

Return type:

yggdrasil.serialize.YAMLSerialize.encode_yaml(obj, fd=None, indent=None, sorted_dict_type=None, sort_keys=True, **kwargs)[source]¶

Encode a Python object in YAML format.

Parameters:

obj (object) – Python object to encode.
fd (file, optional) – File descriptor for file that encoded object should be written to. Defaults to None and string is returned.
indent (int, str, optional) – Indentation for new lines in encoded string. Defaults to None.
sort_keys (bool, optional) – If True, dictionaries will be sorted alphabetically by key. Defaults to True.
**kwargs – Additional keyword arguments are passed to yaml.dump.

Returns:

Encoded object.

Return type:

str, bytes

Module contents¶

yggdrasil.drivers package¶

Submodules¶

yggdrasil.drivers.BuildModelDriver module¶

class yggdrasil.drivers.BuildModelDriver.BuildModelDriver(*args, **kwargs)[source]¶

Bases: CompiledModelDriver

Class for running build file compiled drivers.

Parameters:

name (str) – Driver name.
args (str, list) – Executable that should be created (make target) and any arguments for the executable.
target (str, optional) – Make target that should be built to create the model executable. Defaults to None.
target_language (str, optional) – Language that the target is written in. Defaults to None and will be set based on the source files provided.
target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.
target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.
target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].
target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].
env_compiler (str, optional) – Environment variable where the compiler executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.
env_compiler_flags (str, optional) – Environment variable where the compiler flags should be stored (including those required to compile against the yggdrasil interface). If not provided, this will be determined by the target language driver.
env_linker (str, optional) – Environment variable where the linker executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.
env_linker_flags (str, optional) – Environment variable where the linker flags should be stored (including those required to link against the yggdrasil interface). If not provided, this will be determined by the target language driver.
**kwargs – Additional keyword arguments are passed to parent class.

buildfile¶

Path to file containing build instructions.

Type:: str

builddir¶

Path to directory where build products will be saved.

Type:: str

sourcedir¶

Path to directory where source files are located.

Type:: str

compiledir¶

Path to directory where compilation call should be made from. Defaults to working_dir.

Type:: str

target¶

Name of executable that should be created and called.

Type:: str

target_language¶

Language that the target is written in.

Type:: str

target_language_driver¶

Language driver for the target language.

Type:: ModelDriver

target_compiler¶

Compilation tool that should be used to compile the target language.

Type:: str

target_linker¶

Compilation tool that should be used to link the target language.

Type:: str

target_compiler_flags¶

Compilation flags that should be passed to the target language compiler.

Type:: list

target_linker_flags¶

Linking flags that should be passed to the target language linker.

Type:: list

env_compiler¶

Compiler environment variable.

Type:: str

env_compiler_flags¶

Compiler flags environment variable.

Type:: str

env_linker¶

Linker environment variable.

Type:: str

env_linker_flags¶

Linker flags environment variable.

Type:: str

Class Attributes:

built_where_called (bool): If True, it is assumed that compilation: output will be saved in the same directory from which the compilation command is issued.

Raises:: RuntimeError – If neither the IPC or ZMQ C libraries are available.

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

allow_parallel_build = False¶

base_languages = ['c']¶

buildfile_base = None¶

built_where_called = False¶

classmethod call_compiler(src, **kwargs)[source]¶

Compile a source file into an executable or linkable object file, checking for errors.

Parameters:

src (str) – Full path to source file.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Full path to compiled source.

Return type:

cleanup()[source]¶: Remove compiled executable.

compile_dependencies_instance(*args, **kwargs)[source]¶: Compile dependencies specifically for this instance.

compile_model(**kwargs)[source]¶

Compile model executable(s).

Parameters:: **kwargs – Keyword arguments are passed on to the parent class’s method.

executable_type = 'build'¶

classmethod fix_path(path, for_env=False)[source]¶

Update a path.

Parameters:

path (str) – Path that should be formatted.
for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.

Returns:

Updated path.

Return type:

full_language = False¶

classmethod get_buildfile_lock(**kwargs)[source]¶: Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

classmethod get_language_for_buildfile(buildfile, target=None)[source]¶

Determine the target language based on the contents of a build file.

Parameters:

buildfile (str) – Full path to the build configuration file.
target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_language_for_source(fname=None, buildfile=None, languages=None, early_exit=False, call_base=False, **kwargs)[source]¶

Determine the language that can be used with the provided source file(s). If more than one language applies to a set of multiple files, the language that applies to the most files is returned.

Parameters:

fname (str, list) – The full path to one or more files. If more than one is provided, they are iterated over.
buildfile (str, optional) – Full path to the build configuration file. Defaults to None and will be searched for.
languages (list, optional) – The list of languages that are acceptable. Defaults to None and any language will be acceptable.
early_exit (bool, optional) – If True, the first language identified will be returned if fname is a list of files. Defaults to False.
source_dir (str, optional) – Full path to the directory containing the source files. Defaults to None and is determiend from fname.
buildfile – Full path to the build configuration file. Defaults to None and will be searched for.
target (str, optional) – The build target. Defaults to None.
call_base (bool, optional) – If True, the base class’s method is called directly. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The language that can operate on the specified file.

Return type:

classmethod get_source_dir(fname=None, source_dir=None)[source]¶

classmethod get_target_language_info(target_language_driver=None, target_language='c', target_compiler=None, target_compiler_flags=None, target_linker=None, target_linker_flags=None, logging_level=None, without_wrapper=False, compiler_flag_kwargs=None, linker_flag_kwargs=None, **kwargs)[source]¶

Get a dictionary of information about language compilation tools.

Parameters:

target_language_driver (ModelDriver, optional) – Driver associated with the target language. If not provided, one will be created based on ‘target_language’.
target_language (str, optional) – Language to get info for. Defaults to ‘c’.
target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.
target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.
target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].
target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].
logging_level (int, optional) – The numeric logging level that should be passed as a definition. Defaults to None and is ignored.
without_wrapper (bool, optional) – If True, the returned info will be updated for compilation when a build wrapper was not created. Defaults to False.
compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.
linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.
**kwargs – Additional keyword arguments are added to the output dictionary.

Returns:

Information about language compilers and linkers.

Return type:

is_build_tool = True¶

classmethod is_source_file(fname)[source]¶

Determine if the provided file name points to a source files for the associated programming language by checking the extension.

Parameters:: fname (str) – Path to file.
Returns:: True if the provided file is a source file, False otherwise.
Return type:: bool

parse_arguments(args, **kwargs)[source]¶

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

set_env(**kwargs)[source]¶

Get environment variables that should be set for the model process.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method.
Returns:: Environment variables for the model process.
Return type:: dict

classmethod set_env_compiler(language_info=None, **kwargs)[source]¶

Get environment variables that should be set for the compilation process.

Parameters:

language_info (dict) – Language compilation tool information.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

set_target_language()[source]¶

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns:: Name of language.
Return type:: str

sourcedir_as_sourcefile = False¶

property target_language_info¶

Information about the underlying language.

Type:: dict

class yggdrasil.drivers.BuildModelDriver.BuildToolBase(**kwargs)[source]¶

Bases: CompilerBase

Base class for build tools.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = None¶

classmethod get_flags(**kwargs)[source]¶

Get compilation flags, replacing outfile with target.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method.
Returns:: Compiler flags.
Return type:: list

classmethod get_tool_suffix()[source]¶

Get the string that should be added to tool products based on the tool used.

Returns:

Suffix that should be added to tool products to indicate the: tool used.

Return type:

is_build_tool = True¶

yggdrasil.drivers.CMakeModelDriver module¶

class yggdrasil.drivers.CMakeModelDriver.CMakeBuilder(**kwargs)[source]¶

Bases: LinkerBase

CMake build tool.

classmethod call(*args, **kwargs)[source]¶: Print contents of CMakeCache.txt before raising error.

default_executable = 'cmake'¶

default_flags = []¶

executable_ext = ''¶

classmethod extract_kwargs(kwargs, **kwargs_ex)[source]¶

Extract linker kwargs, leaving behind just compiler kwargs.

Parameters:

kwargs (dict) – Keyword arguments passed to the compiler that should be sorted into kwargs used by either the compiler or linker or both. Keywords that are not used by the compiler will be removed from this dictionary.
**kwargs_ex – Additional keyword arguments are passed to the parent class’s method.

Returns:

Keyword arguments that should be passed to the linker.

Return type:

flag_options = {'builddir': {'key': '--build', 'position': 0}, 'configuration': '--config', 'target': '--target'}¶

classmethod get_executable_command(args, **kwargs)[source]¶

Determine the command required to run the tool using the specified arguments and options.

Parameters:

args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Output to stdout from the command execution.

Return type:

classmethod get_flags(target=None, builddir=None, **kwargs)[source]¶

Get a list of build flags for building a project using cmake.

Parameters:

target (str, optional) – Target that should be built. Defaults to to None and is ignored.
builddir (str, optional) – Directory containing the build tree. Defaults to None and is set based on outfile is provided or cls.default_builddir if not. Defaults to ‘.’ (which will be the current working directory).
**kwargs – Additional keyword arguments are ignored.

Returns:

Linker flags.

Return type:

classmethod get_output_file(obj, target=None, builddir=None, **kwargs)[source]¶

Determine the appropriate output file that will result when bulding a given directory.

Parameters:

obj (str) – Directory being built or a file in the directory being built.
target (str, optional) – Target that will be used to create the output file. Defaults to None. Target is required in order to determine the name of the file that will be created.
builddir (str, optional) – Directory where build tree should be created. Defaults to None and obj will used (if its a directory) or the directory containing obj will be used (if its a file).
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Full path to file that will be produced.

Return type:

Raises:

RuntimeError – If target is None.

is_gnu = False¶

languages = ['cmake']¶

library_ext = '.so'¶

output_key = None¶

tool_suffix_format = ''¶

toolname = 'cmake'¶

class yggdrasil.drivers.CMakeModelDriver.CMakeConfigure(**kwargs)[source]¶

Bases: BuildToolBase

CMake configuration tool.

add_libraries = False¶

classmethod append_product(products, src, new, **kwargs)[source]¶

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters:

products (list) – List of of existing products that new product should be appended to.
src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).
new (str) – New product that should be appended to the list.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = 'cmake'¶

classmethod call(args, **kwargs)[source]¶

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters:

args (list) – The arguments that should be passed to the tool.
**kwargs – Additional keyword arguments are passed to the parent class’s method and the associated linker/archiver’s call method if dont_link is False.

Returns:

Output to stdout from the command execution if skip_flags is: True, produced file otherwise.

Return type:

combine_with_linker = False¶

compile_only_flag = None¶

classmethod create_include(fname, target, driver=None, compiler=None, compiler_flags=None, linker=None, linker_flags=None, library_flags=None, internal_library_flags=None, configuration='Release', verbose=False, **kwargs)[source]¶

Create CMakeList include file with necessary includes, definitions, and linker flags.

Parameters:

fname (str) – File where the include file should be saved.
target (str) – Target that links should be added to.
driver (CompiledModelDriver) – Driver for the language being compiled.
compiler (CompilerBase) – Compiler that should be used to generate the list of compilation flags.
compile_flags (list, optional) – Additional compile flags that should be set. Defaults to [].
linker (LinkerBase) – Linker that should be used to generate the list of compilation flags.
linker_flags (list, optional) – Additional linker flags that should be set. Defaults to [].
library_flags (list, optional) – List of library flags to add. Defaults to [].
internal_library_flags (list, optional) – List of library flags associated with yggdrasil libraries. Defaults to [].
configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’. Only used on Windows to determine the standard library.
verbose (bool, optional) – If True, the contents of the created file are displayed. Defaults to False.
**kwargs – Additional keyword arguments are ignored.

Returns:

Lines that should be added before the executable is: defined in the CMakeLists.txt (e.g. LINK_DIRECTORIES commands).

Return type:

Raises:

ValueError – If a linker or compiler flag cannot be interpreted.

default_archiver = False¶

default_builddir = '.'¶

default_executable = 'cmake'¶

default_flags = []¶

classmethod fix_path(x, is_gnu=False)[source]¶: Fix paths so that they conform to the format expected by the OS and/or build tool.

flag_options = {'builddir': '-B%s', 'configuration': '-DCMAKE_BUILD_TYPE=%s', 'definitions': '-D%s', 'generator': '-G%s', 'platform': '-A%s', 'sourcedir': '', 'toolset': '-T%s'}¶

classmethod generator(return_default=False, default=None, **kwargs)[source]¶

Determine the generator that should be used.

Parameters:

return_default (bool, optional) – If True, the default generator will be returned even if the environment variable is set. Defaults to False.
default (str, optional) – Value that should be returned if a generator cannot be located. Defaults to None.
**kwargs – Keyword arguments are passed to cls.call.

Returns:

Name of the generator.

Return type:

classmethod generator2toolset(generator)[source]¶

Determine the toolset string option that corresponds to the provided generator name.

Parameters:

generator (str) – Name of the generator.

Returns:

Name of the toolset.

Return type:

Raises:

NotImplementedError – If the platform is not windows.
ValueError – If the generator is not a flavor of Visual Studio.
ValueError – If a tool set cannot be located for the specified generator.

classmethod get_executable_command(args, **kwargs)[source]¶

Determine the command required to run the tool using the specified arguments and options.

Parameters:

args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Output to stdout from the command execution.

Return type:

classmethod get_flags(sourcedir='.', builddir=None, target_compiler=None, target_linker=None, **kwargs)[source]¶

Get a list of configuration/generation flags.

Parameters:

sourcedir (str, optional) – Directory containing the source files to be compiled and the target CMakeLists.txt file. Defaults to ‘.’ (the current working directory).
builddir (str, optional) – Directory that will contain the build tree. Defaults to ‘.’ (this current working directory).
target_compiler (str, optional) – Compiler that should be used by cmake. Defaults to None and the default for the target language will be used.
target_linker (str, optional) – Linker that should be used by cmake. Defaults to None and the default for the target language will be used.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Compiler flags.

Return type:

Raises:

RuntimeError – If dont_link is True and the provide outfile and builddir keyword arguments point to conflicting paths.
ValueError – If ‘include_dirs’ is set (‘sourcedir’ should be used for cmake to specify the location of the source).

classmethod get_output_file(src, dont_link=False, dont_build=None, sourcedir=None, builddir=None, working_dir=None, **kwargs)[source]¶

Determine the appropriate output file or directory that will result when configuring/building a given source directory.

Parameters:

src (str) – Directory containing source files being compiled.
dont_link (bool, optional) – If True, the result assumes that the source is just compiled and not linked. If False, the result will be the final result after linking.
dont_build (bool, optional) – Alias for dont_link. If not None, this keyword overrides the value of dont_link. Defaults to None.
sourcedir (str, optional) – Directory where sources files are located. Defaults to None. If None, src will be used to determine the value.
builddir (str, optional) – Directory where build tree should be created. Defaults to None. If None, sourcedir will be used.
working_dir (str, optional) – Working directory where output file should be located. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are ignored unless dont_link is False; then they are passed to get_linker_output_file

Returns:

Full path to file that will be produced.

Return type:

is_gnu = False¶

is_linker = False¶

languages = ['cmake']¶

output_key = None¶

product_files = ['Makefile', 'CMakeCache.txt', 'cmake_install.cmake', 'CMakeFiles']¶

remove_product_exts = ['CMakeFiles']¶

toolname = 'cmake'¶

class yggdrasil.drivers.CMakeModelDriver.CMakeModelDriver(*args, **kwargs)[source]¶

Bases: BuildModelDriver

Class for running cmake compiled drivers. Before running the cmake command, the cmake commands for setting the necessary compiler & linker flags for the interface’s C/C++ library are written to a file called ‘ygg_cmake.txt’ that should be included in the CMakeLists.txt file (after the target executable has been added).

Parameters:

name (str) – Driver name.
args (str, list) – Executable that should be created (cmake target) and any arguments for the executable.
sourcedir (str, optional) – Source directory to call cmake on. If not provided it is set to working_dir. This should be the directory containing the CMakeLists.txt file. It can be relative to working_dir or absolute.
builddir (str, optional) – Directory where the build should be saved. Defaults to <sourcedir>/build. It can be relative to working_dir or absolute.
configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’.
**kwargs – Additional keyword arguments are passed to parent class.

sourcedir¶

Source directory to call cmake on.

Type:: str

add_libraries¶

If True, interface libraries and dependency libraries are added using CMake’s ADD_LIBRARY directive. If False, interface libraries are found using FIND_LIBRARY.

Type:: bool

configuration¶

Build type/configuration that should be built. This is only used on Windows.

Type:: str

Raises:: RuntimeError – If neither the IPC or ZMQ C libraries are available.

add_libraries = False¶

buildfile_base = 'CMakeLists.txt'¶

property buildfile_orig¶

Full path to where the original CMakeLists.txt file will be stored during compilation of the modified file.

Type:: str

property buildfile_ygg¶

Full path to the verison of the CMakeLists.txt that has been updated w/ yggdrasil compilation flags.

Type:: str

cfg = <yggdrasil.config.YggConfigParser object>¶

compile_model(target=None, **kwargs)[source]¶

Compile model executable(s) and appends any products produced by the compilation that should be removed after the run is complete.

Parameters:

target (str, optional) – Target to build.
**kwargs – Keyword arguments are passed on to the call_compiler method.

default_archiver = None¶

default_archiver_flags = None¶

default_compiler = 'cmake'¶

default_compiler_flags = None¶

default_linker = 'cmake'¶

default_linker_flags = None¶

classmethod fix_path(path, for_env=False, **kwargs)[source]¶

Update a path.

Parameters:

path (str) – Path that should be formatted.
for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Updated path.

Return type:

classmethod get_language_for_buildfile(buildfile, target=None)[source]¶

Determine the target language based on the contents of a build file.

Parameters:

buildfile (str) – Full path to the build configuration file.
target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_target_language_info(target_compiler_flags=None, target_linker_flags=None, compiler_flag_kwargs=None, linker_flag_kwargs=None, without_wrapper=False, **kwargs)[source]¶

Get a dictionary of information about language compilation tools.

Parameters:

target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].
target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].
compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.
linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.
**kwargs – Keyword arguments are passed to the parent class’s method.

Returns:

Information about language compilers and linkers.

Return type:

inverse_type_map = None¶

language = 'cmake'¶

parse_arguments(args, **kwargs)[source]¶

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod prune_sh_gcc(path, gcc)[source]¶

Remove instances of sh.exe from the path that are not associated with the selected gcc compiler. This can happen on windows when rtools or git install a version of sh.exe that is added to the path before the compiler.

Parameters:

path (str) – Contents of the path variable.
gcc (str) – Full path to the gcc executable.

Returns:

Modified path that removes the extra instances: of sh.exe.

Return type:

sourcedir_as_sourcefile = True¶

classmethod update_compiler_kwargs(**kwargs)[source]¶

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Keyword arguments for a get_flags method providing compiler: flags.

Return type:

use_env_vars = False¶

write_wrappers(**kwargs)[source]¶

Write any wrappers needed to compile and/or run a model.

Parameters:: **kwargs – Keyword arguments are passed to the parent class’s method.
Returns:: Full paths to any created wrappers.
Return type:: list

yggdrasil.drivers.CModelDriver module¶

class yggdrasil.drivers.CModelDriver.ARArchiver(**kwargs)[source]¶

Bases: ArchiverBase

Archiver class for ar tool.

asan_flags = []¶

compatible_toolsets = ['gnu', 'llvm']¶

default_executable = 'ar'¶

default_executable_env = 'AR'¶

default_flags = []¶

default_flags_env = None¶

is_gnu = True¶

languages = ['c', 'c++', 'fortran']¶

library_ext = '.a'¶

output_first_library = True¶

output_key = ''¶

search_path_envvar = ['LIBRARY_PATH']¶

shared_library_flag = None¶

static_library_flag = 'rcs'¶

toolname = 'ar'¶

toolset = 'gnu'¶

class yggdrasil.drivers.CModelDriver.CCompilerBase(**kwargs)[source]¶

Bases: CompilerBase

Base class for C compilers.

classmethod call(args, **kwargs)[source]¶: Call the compiler with the provided arguments. For yggdrasil C models will always be linked using the C++ linker since some parts of the interface library are written in C++.

default_executable_env = 'CC'¶

default_flags = ['-g', '-Wall']¶

default_flags_env = 'CFLAGS'¶

classmethod get_search_path(*args, **kwargs)[source]¶

Determine the paths searched by the tool for external library files.

Parameters:

*args – Additional arguments are passed to the parent class’s method.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

List of paths that the tools will search.

Return type:

languages = ['c']¶

linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH']}¶

search_path_envvar = ['C_INCLUDE_PATH']¶

search_path_flags = ['-E', '-v', '-xc', '/dev/null']¶

search_regex = ['(?:#include <...> search starts here:)|(?: ([^\\n]+?)(?: \$framework directory\$)?)\\n']¶

search_regex_begin = '#include "..." search starts here:'¶

search_regex_end = 'End of search list.'¶

classmethod set_env(*args, **kwargs)[source]¶

Set environment variables required for compilation.

Parameters:

*args – Arguments are passed to the parent class’s method.
**kwargs – Keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

class yggdrasil.drivers.CModelDriver.CModelDriver(*args, **kwargs)[source]¶

Bases: CompiledModelDriver

Class for running C models.

classmethod add_extra_vars(direction, x)[source]¶

Add extra variables required for communication.

Parameters:

direction (str) – Direction of channel (‘input’ or ‘output’).
x (dict) – Dictionary describing the variable.

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration.

classmethod allows_realloc(var)[source]¶

Determine if a variable allows the receive call to perform realloc.

Parameters:: var (dict) – Dictionary of variable properties.
Returns:: True if the variable allows realloc, False otherwise.
Return type:: bool

brackets = ('{', '}')¶

classmethod call_linker(obj, language=None, **kwargs)[source]¶

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters:

obj (list) – Object files that should be linked.
language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.
**kwargs – Additional keyword arguments are passed to run_executable.

Returns:

Full path to compiled source.

Return type:

cfg = <yggdrasil.config.YggConfigParser object>¶

classmethod configure(cfg, macos_sdkroot=None, vcpkg_dir=None, **kwargs)[source]¶

Add configuration options for this language. This includes locating any required external libraries and setting option defaults.

Parameters:

cfg (YggConfigParser) – Config class that options should be set for.
macos_sdkroot (str, optional) – Full path to the root directory for the MacOS SDK that should be used. Defaults to None and is ignored.
vcpkg_dir (str, optional) – Full path to the root directory containing a vcpkg installation. This should be the directory that contains the vcpkg executable and any packages installed by vcpkg (in subdirectories). Defaults to None and is ignored.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

default_archiver = 'ar'¶

default_archiver_flags = None¶

default_compiler = 'gcc'¶

default_compiler_flags = None¶

default_linker = 'gcc'¶

default_linker_flags = None¶

external_libraries = {'czmq': {'include': 'czmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libczmq.so'}, 'numpy': {'for_python_api': True, 'include': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h', 'language': 'c', 'libtype': 'header_only'}, 'python': {'for_python_api': True, 'include': '/opt/hostedtoolcache/Python/3.8.17/x64/include/python3.8/Python.h', 'language': 'c', 'libtype': 'shared', 'shared': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/libpython3.8.so', 'standard': True}, 'rapidjson': {'include': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/rapidjson/include/rapidjson/rapidjson.h', 'language': 'c', 'libtype': 'header_only'}, 'zmq': {'include': 'zmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libzmq.so'}}¶

classmethod finalize_function_io(direction, x)[source]¶

Finalize info for an input/output channel following function parsing.

Parameters:

direction (str) – Direction of channel (‘input’ or ‘output’)
x (dict) – Channel info.

classmethod format_function_param(key, default=None, **kwargs)[source]¶

Return the formatted version of the specified key.

Parameters:

key (str) – Key in cls.function_param mapping that should be formatted.
default (str, optional) – Format that should be returned if key is not in cls.function_param. Defaults to None.
**kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns:

Formatted string.

Return type:

Raises:

NotImplementedError – If key is not in cls.function_param and default is not set.

function_param = {'and': '&&', 'assign': '{name} = {value};', 'assign_copy': 'memcpy({name}, {value}, {N}*sizeof({native_type}));', 'block_end': '}', 'break': 'break;', 'comment': '//', 'copy_any': 'copy_generic_into(&{name}, {value});', 'copy_array': 'copy_generic_into(&{name}, {value});', 'copy_class': '{name} = copy_python({value});', 'copy_function': '{name} = copy_python({value});', 'copy_instance': 'copy_generic_into(&{name}, {value});', 'copy_obj': '{name} = copy_obj({value});', 'copy_object': 'copy_generic_into(&{name}, {value});', 'copy_ply': '{name} = copy_ply({value});', 'copy_schema': 'copy_generic_into(&{name}, {value});', 'declare': '{type_name} {variable};', 'error': 'printf("{error_msg}\\n"); return -1;', 'exec_begin': 'int main() {', 'exec_end': ' return 0;\n}', 'exec_prefix': '#include <stdbool.h>', 'false': 'false', 'false_flag': '0', 'flag_cond': '{flag_var} >= 0', 'for_begin': 'for ({iter_var} = {iter_begin}; {iter_var} < {iter_end}; {iter_var}++) {{', 'fprintf': 'printf("{message}\\n", {variables});', 'free': 'if ({variable} != NULL) {{ free({variable}); {variable} = NULL; }}', 'free_any': 'free_generic({variable});', 'free_array': 'free_json_array({variable});', 'free_class': 'destroy_python({variable});', 'free_function': 'destroy_python({variable});', 'free_instance': 'free_generic({variable});', 'free_obj': 'free_obj({variable});', 'free_object': 'free_json_object({variable});', 'free_ply': 'free_ply({variable});', 'free_schema': 'free_schema({variable});', 'function_def_begin': '{output_type} {function_name}({input_var}) {{', 'function_def_regex': '(?P<flag_type>.+?)\\s*{function_name}\\s*\$(?P<inputs>(?:[^{{])*?)\$\\s*\\{{(?P<body>(?:.*?\\n?)*?)(?:(?:return *(?P<flag_var>.+?)?;(?:.*?\\n?)*?\\}})|(?:\\}}))', 'if_begin': 'if ({cond}) {{', 'if_elif': '}} else if ({cond}) {{', 'if_else': '}} else {{', 'import': '#include "{filename}"', 'indent': ' ', 'index': '{variable}[{index}]', 'init_any': 'init_generic()', 'init_array': 'init_json_array()', 'init_class': 'init_python()', 'init_function': 'init_python()', 'init_instance': 'init_generic()', 'init_obj': 'init_obj()', 'init_object': 'init_json_object()', 'init_ply': 'init_ply()', 'init_schema': 'init_schema()', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_from_schema': 'create_dtype_from_schema("{schema}", {use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray_arr("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': '{channel} = yggInputType("{channel_name}", {channel_type});', 'inputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(\$(?P<name_ptr>\\*+)?)?(?P<name>.+?)(?(4)(?:\$))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'interface': '#include "{interface_library}"', 'line_end': ';', 'not': '!', 'not_flag_cond': '{flag_var} < 0', 'null': 'NULL', 'output': '{channel} = yggOutputType("{channel_name}", {channel_type});', 'outputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(?P<name>.+?)(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'print': 'printf("{message}\\n");', 'print_any': 'display_generic({object});', 'print_array': 'display_json_array({object});', 'print_bytes': 'printf("%s\\n", {object});', 'print_class': 'display_python({object});', 'print_complex': 'print_complex({object});', 'print_float': 'printf("%f\\n", {object});', 'print_function': 'display_python({object});', 'print_instance': 'display_generic({object});', 'print_int': 'printf("%i\\n", {object});', 'print_obj': 'display_obj({object});', 'print_object': 'display_json_object({object});', 'print_ply': 'display_ply({object});', 'print_schema': 'display_schema({object});', 'print_string': 'printf("%s\\n", {object});', 'print_uint': 'printf("%u\\n", {object});', 'print_unicode': 'printf("%s\\n", {object});', 'python_init': 'init_python_API();', 'quote': '"', 'recv_function': 'yggRecvRealloc', 'recv_heap': 'yggRecvRealloc', 'recv_stack': 'yggRecv', 'return': 'return {output_var};', 'send_function': 'yggSend', 'true': 'true', 'true_flag': '1', 'while_begin': 'while ({cond}) {{'}¶

classmethod get_json_type(native_type)[source]¶

Get the JSON type from the native language type.

Parameters:: native_type (str) – The native language type.
Returns:: The JSON type.
Return type:: str, dict

classmethod get_name_declare(var)[source]¶

Determine the name that should be used for declaration.

Parameters:: var (str, dict) – Name of variable or dictionary of information.
Returns:: Modified name for declaration.
Return type:: str

classmethod get_native_type(const=False, **kwargs)[source]¶

Get the native type.

Parameters:

type (str, optional) – Name of yggdrasil extended JSON type or JSONSchema dictionary defining a datatype.
const (bool, optional) – If True, the native type will be marked as constant. Defaults to False.
**kwargs – Additional keyword arguments may be used in determining the precise declaration that should be used.

Returns:

The native type.

Return type:

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

include_channel_obj = True¶

classmethod input2output(var)[source]¶

Perform conversion necessary to turn a variable extracted from a function definition from an input to an output.

Parameters:: var (dict) – Variable definition.
Returns:: Updated variable definition.
Return type:: dict

interface_dependencies = ['rapidjson']¶

interface_directories = ['/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C']¶

interface_library = 'ygg'¶

interface_map = {'call': 'flag = rpcCall({channel_obj}, {outputs}, {inputs})', 'client': 'yggRpcClientType("{channel_name}", {datatype_out}, {datatype_in})', 'import': '#include "YggInterface.h"', 'input': 'yggInputType("{channel_name}", {datatype})', 'output': 'yggOutputType("{channel_name}", {datatype})', 'recv': 'flag = yggRecv({channel_obj}, {input_refs})', 'send': 'flag = yggSend({channel_obj}, {outputs})', 'server': 'yggRpcServerType("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'yggTimesync("{channel_name}", "{time_units}")'}¶

internal_libraries = {'datatypes': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/datatypes', 'external_dependencies': ['rapidjson', 'python', 'numpy', 'm'], 'include_dirs': [], 'internal_dependencies': ['regex'], 'language': 'c++', 'libtype': 'object'}, 'regex': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_posix': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_win32': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c++', 'libtype': 'object', 'source': 'regex_win32.cpp'}, 'ygg': {'compiler_flags': ['-fPIC'], 'external_dependencies': ['rapidjson', 'python', 'numpy', 'm'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'internal_dependencies': ['regex', 'datatypes'], 'language': 'c', 'linker_language': 'c++', 'source': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/YggInterface.c'}}¶

inverse_type_map = None¶

is_typed = True¶

language = 'c'¶

language_ext = ['.c', '.h']¶

locked_buildfile = 'datatypes.o'¶

classmethod output2input(var, in_definition=True)[source]¶

Perform conversion necessary to turn an output variable into an corresponding input that can be used to format a function definition.

Parameters:

var (dict) – Variable definition.
in_definition (bool, optional) – If True, the returned dictionary corresponds to an input variable in a function definition. If False, the returned value will correspond to an input to a function. Defaults to True.

Returns:

Updated variable definition.

Return type:

outputs_in_inputs = True¶

classmethod parse_var_definition(io, value, **kwargs)[source]¶

Extract information about input/output variables from a string definition.

Parameters:

io (str) – Description of variables contained in the provided string. Must be ‘inputs’ or ‘outputs’.
value (str) – String containing one or more variable definitions.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

List of information about the variables contained in: the provided string.

Return type:

Raises:

AssertionError – If io is not ‘inputs’ or ‘outputs’.
NotImplementedError – If the def_regex for the specified io is not defined.

classmethod prepare_output_variables(vars_list, in_definition=False, in_inputs=False, for_yggdrasil=False)[source]¶

Concatenate a set of output variables such that it can be passed as a single string to the function_call parameter.

Parameters:

vars_list (list) – List of variable names to concatenate as output from a function call.
in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying output variables in a function definition. Defaults to False.
in_inputs (bool, optional) – If True, the output variables should be formated to be included as input variables. Defaults to False.
for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns:

Concatentated variables list.

Return type:

classmethod prepare_variables(vars_list, in_definition=False, for_yggdrasil=False)[source]¶

Concatenate a set of input variables such that it can be passed as a single string to the function_call parameter.

Parameters:

vars_list (list) – List of variable dictionaries containing info (e.g. names) that should be used to prepare a string representing input/output to/from a function call.
in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying variables in a function definition. Defaults to False.
for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns:

Concatentated variables list.

Return type:

classmethod requires_length_var(var)[source]¶

Determine if a variable requires a separate length variable.

Parameters:: var (dict) – Dictionary of variable properties.
Returns:: True if a length variable is required, False otherwise.
Return type:: bool

classmethod requires_shape_var(var)[source]¶

Determine if a variable requires a separate shape variable.

Parameters:: var (dict) – Dictionary of variable properties.
Returns:: True if a shape variable is required, False otherwise.
Return type:: bool

classmethod set_env_class(**kwargs)[source]¶

Set environment variables that are instance independent.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.
Returns:: Environment variables for the model process.
Return type:: dict

standard_libraries = ['m']¶

supported_comm_options = {'ipc': {'platforms': ['MacOS', 'Linux']}, 'zmq': {'libraries': ['zmq', 'czmq']}}¶

supported_comms = ['ipc', 'zmq']¶

type_class_map = {}¶

type_map = {'1darray': '*', 'any': 'generic_t', 'array': 'json_array_t', 'boolean': 'bool', 'bytes': 'string_t', 'class': 'python_class_t', 'comm': 'comm_t*', 'complex': 'complex_X', 'dtype': 'dtype_t*', 'flag': 'int', 'float': 'double', 'function': 'python_function_t', 'instance': 'python_instance_t', 'int': 'intX_t', 'integer': 'int', 'length': 'size_t', 'ndarray': '*', 'null': 'void*', 'number': 'double', 'obj': 'obj_t', 'object': 'json_object_t', 'ply': 'ply_t', 'schema': 'schema_t', 'string': 'char*', 'uint': 'uintX_t', 'unicode': 'unicode_t'}¶

classmethod update_ld_library_path(env, paths_to_add=None, add_to_front=False, add_libpython_dir=False, toolname=None, env_var=None, **kwargs)[source]¶

Update provided dictionary of environment variables so that LD_LIBRARY_PATH includes the interface directory containing the interface libraries.

Parameters:

env (dict) – Dictionary of enviroment variables to be updated.
paths_to_add (list, optional) – Paths that should be added. If not provided, defaults to [cls.get_language_dir()].
add_to_front (bool, optional) – If True, new paths are added to the front, rather than the end. Defaults to False.
add_libpython_dir (bool, optional) – If True, the directory containing the Python C library will be added. Defaults to False.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
env_var (str, optional) – Environment variable where the paths should be added. Defaults to None and is only set for linux (LD_LIBRARY_PATH) and windows (PATH).
**kwargs – Additional keyword arguments are ignored.

Returns:

Updated dictionary of environment variables.

Return type:

classmethod update_python_path(env)[source]¶

Update provided dictionary of environment variables so that PYTHONPATH and PYTHONHOME are set as needed (primarily on windows).

Parameters:: env (dict) – Dictionary of enviroment variables to be updated.
Returns:: Updated dictionary of environment variables.
Return type:: dict

classmethod write_assign_to_output(dst_var, src_var, outputs_in_inputs=False, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]¶

Write lines assigning a value to an output variable.

Parameters:

dst_var (str, dict) – Name or information dictionary for variable being assigned to.
src_var (str, dict) – Name or information dictionary for value being assigned to dst_var.
outputs_in_inputs (bool, optional) – If True, outputs are passed as input parameters. In some languages, this means that a pointer or reference is passed (e.g. C) and so the assignment should be to the memory indicated rather than the variable. Defaults to False.
dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.
use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines achieving assignment.

Return type:

classmethod write_declaration(var, **kwargs)[source]¶

Return the lines required to declare a variable with a certain type.

Parameters:

var (dict, str) – Name or information dictionary for the variable being declared.
**kwargs – Addition keyword arguments are passed to the parent class’s method.

Returns:

The lines declaring the variable.

Return type:

classmethod write_free(var, **kwargs)[source]¶

Return the lines required to free a variable with a certain type.

Parameters:

var (dict, str) – Name or information dictionary for the variable being declared.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The lines freeing the variable.

Return type:

classmethod write_function_def(function_name, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]¶

Write a function definition.

Parameters:

function_name (str) – Name fo the function being defined.
dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.
use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines completing the function call.

Return type:

Raises:

ValueError – If outputs_in_inputs is not True and more than one output variable is specified.

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]¶

Get the lines necessary to initialize an array for iteration output.

Parameters:

var (dict, str) – Name or information dictionary for the variable being initialized.
value (str, optional) – Value that should be assigned to the variable.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

The lines initializing the variable.

Return type:

classmethod write_model_function_call(model_function, flag_var, inputs, outputs, **kwargs)[source]¶

Write lines necessary to call the model function.

Parameters:

model_function (str) – Handle of the model function that should be called.
flag_var (str) – Name of variable that should be used as a flag.
inputs (list) – List of dictionaries describing inputs to the model.
outputs (list) – List of dictionaries describing outputs from the model.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines required to carry out a call to a model function in: this language.

Return type:

classmethod write_model_recv(channel, recv_var, **kwargs)[source]¶

Write a model receive call include checking the return flag.

Parameters:

channel (str) – Name of variable that the channel being received from was stored in.
recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines required to carry out a receive call in this language.

Return type:

classmethod write_print_output_var(var, in_inputs=False, **kwargs)[source]¶

Get the lines necessary to print an output variable in this language.

Parameters:

var (dict) – Variable information.
in_inputs (bool, optional) – If True, the output variable is passed in as an input variable to be populated. Defaults to False.
**kwargs – Additional keyword arguments are passed to write_print_var.

Returns:

Lines printing the specified variable.

Return type:

class yggdrasil.drivers.CModelDriver.ClangCompiler(**kwargs)[source]¶

Bases: CCompilerBase

Interface class for clang compiler/linker.

asan_flags = ['-fsanitize=address']¶

combine_with_linker = True¶

compatible_toolsets = ['llvm']¶

default_archiver = 'libtool'¶

default_executable = 'clang'¶

default_flags = ['-g', '-Wall']¶

default_linker = 'clang'¶

flag_options = {'definitions': '-D%s', 'include_dirs': '-I%s', 'isysroot': {'key': '-isysroot', 'prepend': True}, 'mmacosx-version-min': '-mmacosx-version-min=%s', 'sysroot': '--sysroot'}¶

classmethod get_flags(*args, **kwargs)[source]¶: Get a list of compiler flags.

is_gnu = False¶

is_linker = False¶

object_tool = 'otool -L'¶

platforms = ['MacOS', 'Linux', 'Windows']¶

preload_envvar = 'DYLD_INSERT_LIBRARIES'¶

toolname = 'clang'¶

toolset = 'llvm'¶

class yggdrasil.drivers.CModelDriver.ClangLinker(**kwargs)[source]¶

Bases: LDLinker

Interface class for clang linker (calls to ld).

aliases = []¶

asan_flags = ['-fsanitize=address', '-shared-libasan']¶

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

compatible_toolsets = ['llvm']¶

default_executable = 'clang'¶

default_flags = []¶

flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': '', 'library_rpath': '-rpath', 'linker-version': '-mlinker-version=%s'}¶

classmethod get_flags(*args, **kwargs)[source]¶: Get a list of linker flags.

is_gnu = False¶

languages = ['c']¶

library_ext = '.so'¶

platforms = ['MacOS', 'Linux', 'Windows']¶

search_path_flags = ['-Xlinker', '-v']¶

search_regex = ['\\t([^\\t\\n]+)\\n']¶

search_regex_begin = 'Library search paths:'¶

classmethod tool_version(**kwargs)[source]¶

Determine the version of this tool.

Parameters:: **kwargs – Keyword arguments are passed to cls.call.
Returns:: Version of the tool.
Return type:: str

toolname = 'clang'¶

toolset = 'llvm'¶

class yggdrasil.drivers.CModelDriver.GCCCompiler(**kwargs)[source]¶

Bases: CCompilerBase

Interface class for gcc compiler/linker.

aliases = ['gnu-cc', 'gnu-gcc']¶

asan_flags = ['-fsanitize=address']¶

combine_with_linker = True¶

compatible_toolsets = ['gnu']¶

default_archiver = 'ar'¶

default_executable = 'gcc'¶

default_flags = ['-g', '-Wall']¶

default_linker = 'gcc'¶

dll2a(dll, dst=None, overwrite=False)[source]¶

Convert a window’s .dll library into a static library.

Parameters:

dll (str) – Full path to .dll library to convert.
dst (str, optional) – Full path to location where the new library should be saved. Defaults to None and will be set based on lib or will be placed in the same directory as dll.
overwrite (bool, optional) – If True, the static file will be created even if it already exists. Defaults to False.

Returns:

Full path to new .a static library.

Return type:

classmethod get_flags(*args, **kwargs)[source]¶: Get a list of compiler flags.

is_gnu = True¶

classmethod is_installed()[source]¶

Determine if this tool is installed by looking for the executable.

Returns:: True if the tool is installed, False otherwise.
Return type:: bool

is_linker = False¶

object_tool = 'ldd'¶

platforms = ['MacOS', 'Linux', 'Windows']¶

preload_envvar = 'LD_PRELOAD'¶

toolname = 'gcc'¶

toolset = 'gnu'¶

class yggdrasil.drivers.CModelDriver.GCCLinker(**kwargs)[source]¶

Bases: LDLinker

Interface class for gcc linker (calls to ld).

aliases = ['gnu-cc', 'gnu-gcc']¶

compatible_toolsets = ['gnu']¶

default_executable = 'gcc'¶

default_flags = []¶

flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': {'allow_duplicate_values': True, 'key': '-l:%s'}, 'library_rpath': '-Wl,-rpath'}¶

is_gnu = True¶

languages = ['c']¶

library_ext = '.so'¶

platforms = ['MacOS', 'Linux', 'Windows']¶

search_path_flags = ['-Xlinker', '--verbose']¶

search_regex = ['SEARCH_DIR\$"=([^"]+)"\$;']¶

toolname = 'gcc'¶

toolset = 'gnu'¶

class yggdrasil.drivers.CModelDriver.LDLinker(**kwargs)[source]¶

Bases: LinkerBase

Linker class for ld tool.

asan_flags = ['-fsanitize=address']¶

default_executable = 'ld'¶

default_executable_env = 'LD'¶

default_flags = []¶

default_flags_env = 'LDFLAGS'¶

classmethod get_flags(*args, **kwargs)[source]¶: Get a list of linker flags.

is_gnu = False¶

languages = ['c']¶

library_ext = '.so'¶

search_path_envvar = ['LIBRARY_PATH', 'LD_LIBRARY_PATH']¶

classmethod tool_version(**kwargs)[source]¶

Determine the version of this tool.

Parameters:: **kwargs – Keyword arguments are passed to cls.call.
Returns:: Version of the tool.
Return type:: str

toolname = 'ld'¶

version_flags = ['-v']¶

class yggdrasil.drivers.CModelDriver.LibtoolArchiver(**kwargs)[source]¶

Bases: ArchiverBase

Archiver class for libtool tool.

asan_flags = []¶

compatible_toolsets = ['clang']¶

default_executable = 'libtool'¶

default_executable_env = 'LIBTOOL'¶

default_flags = []¶

is_gnu = False¶

languages = ['c', 'c++']¶

library_ext = '.a'¶

search_path_envvar = ['LIBRARY_PATH']¶

shared_library_flag = None¶

static_library_flag = '-static'¶

toolname = 'libtool'¶

toolset = 'clang'¶

class yggdrasil.drivers.CModelDriver.MSVCArchiver(**kwargs)[source]¶

Bases: ArchiverBase

Microsoft Visual Studio C Archiver.

compatible_toolsets = ['msvc', 'llvm']¶

default_executable = 'LIB'¶

default_flags = []¶

is_gnu = False¶

languages = ['c', 'c++']¶

library_ext = '.a'¶

output_key = '/OUT:%s'¶

platforms = ['Windows']¶

search_path_envvar = ['LIB']¶

shared_library_flag = None¶

static_library_flag = None¶

toolname = 'LIB'¶

toolset = 'msvc'¶

class yggdrasil.drivers.CModelDriver.MSVCCompiler(**kwargs)[source]¶

Bases: CCompilerBase

Microsoft Visual Studio C Compiler.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

combine_with_linker = True¶

compatible_toolsets = ['msvc']¶

default_archiver = 'LIB'¶

default_executable = 'cl'¶

default_flags = ['/W4', '/Zi', '/FS', '/EHsc', '/bigobj', '/TP', '/nologo', '-D_CRT_SECURE_NO_WARNINGS']¶

default_linker = 'LINK'¶

is_gnu = False¶

linker_attributes = {'default_executable': None, 'default_executable_env': 'LINK', 'default_flags_env': 'LDFLAGS', 'flag_options': {'import_lib': '/IMPLIB:%s', 'library_dirs': '/LIBPATH:%s', 'library_libs': '', 'library_libs_nonstd': ''}, 'output_first': True, 'output_first_library': False, 'output_key': '/OUT:%s', 'search_path_envvar': ['LIB'], 'search_path_flags': None, 'shared_library_flag': '/DLL'}¶

linker_switch = '/link'¶

output_first = True¶

output_key = '/Fo%s'¶

platforms = ['Windows']¶

product_exts = ['.dir', '.ilk', '.pdb', '.sln', '.vcxproj', '.vcxproj.filters', '.exp', '.lib']¶

search_path_envvar = ['INCLUDE']¶

search_path_flags = None¶

classmethod tool_version(**kwargs)[source]¶

Determine the version of this tool.

Parameters:: **kwargs – Keyword arguments are passed to cls.call.
Returns:: Version of the tool.
Return type:: str

toolname = 'cl'¶

toolset = 'msvc'¶

version_flags = []¶

yggdrasil.drivers.CModelDriver.get_OSX_SYSROOT()[source]¶

Determin the path to the OSX SDK.

Returns:

Full path to the SDK directory if one is located. None: otherwise.

Return type:

yggdrasil.drivers.CPPModelDriver module¶

class yggdrasil.drivers.CPPModelDriver.CPPCompilerBase(**kwargs)[source]¶

Bases: CCompilerBase

Base class for C++ compilers.

cpp_std = 'c++14'¶

default_executable = None¶

default_executable_env = 'CXX'¶

default_flags_env = 'CXXFLAGS'¶

default_linker = None¶

classmethod find_standard_flag(flags)[source]¶

Locate the standard flag in a list of flags.

Parameters:: flags (list) – Compilation flags.
Returns:: Index of the standard flag. -1 if not present.
Return type:: int

classmethod handle_standard_flag(flags, skip_standard_flag=False)[source]¶

Add or remove standard flag from a list of flags.

Parameters:

flags (list) – Compilation flags.
skip_standard_flag (bool, optional) – If True, the C++ standard flag will not be added. Defaults to False.

languages = ['c++']¶

search_path_flags = ['-E', '-v', '-xc++', '/dev/null']¶

class yggdrasil.drivers.CPPModelDriver.CPPModelDriver(*args, **kwargs)[source]¶

Bases: CModelDriver

Class for running C++ models.

classmethod add_extra_vars(direction, x)[source]¶

Add extra variables required for communication.

Parameters:

direction (str) – Direction of channel (‘input’ or ‘output’).
x (dict) – Dictionary describing the variable.

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration.

classmethod allows_realloc(var)[source]¶

Determine if a variable allows the receive call to perform realloc.

Parameters:: var (dict) – Dictionary of variable properties.
Returns:: True if the variable allows realloc, False otherwise.
Return type:: bool

base_languages = ['c']¶

cfg = <yggdrasil.config.YggConfigParser object>¶

default_archiver = 'ar'¶

default_archiver_flags = None¶

default_compiler = 'g++'¶

default_compiler_flags = None¶

default_linker = 'g++'¶

default_linker_flags = None¶

dont_declare_channel = True¶

function_param = {'and': '&&', 'assign': '{name} = {value};', 'assign_copy': 'memcpy({name}, {value}, {N}*sizeof({native_type}));', 'block_end': '}', 'break': 'break;', 'comment': '//', 'copy_any': '{name}.CopyFrom({value}, {name}.GetAllocator(), true);', 'copy_array': '{name}.CopyFrom({value}, {name}.GetAllocator(), true);', 'copy_class': '{name} = copy_python({value});', 'copy_function': '{name} = copy_python({value});', 'copy_instance': 'copy_generic_into(&{name}, {value});', 'copy_object': '{name}.CopyFrom({value}, {name}.GetAllocator(), true);', 'copy_schema': '{name}.CopyFrom({value}, {name}.GetAllocator(), true);', 'declare': '{type_name} {variable};', 'error': 'throw "{error_msg}";', 'exec_begin': 'int main() {', 'exec_end': ' return 0;\n}', 'exec_prefix': '#include <iostream>\n#include <exception>\n', 'false': 'false', 'false_flag': '0', 'flag_cond': '{flag_var} >= 0', 'for_begin': 'for ({iter_var} = {iter_begin}; {iter_var} < {iter_end}; {iter_var}++) {{', 'fprintf': 'printf("{message}\\n", {variables});', 'free': 'if ({variable} != NULL) {{ free({variable}); {variable} = NULL; }}', 'free_any': 'free_generic({variable});', 'free_array': 'free_json_array({variable});', 'free_class': 'destroy_python({variable});', 'free_function': 'destroy_python({variable});', 'free_instance': 'free_generic({variable});', 'free_obj': 'free_obj({variable});', 'free_object': 'free_json_object({variable});', 'free_ply': 'free_ply({variable});', 'free_schema': 'free_schema({variable});', 'function_def_begin': '{output_type} {function_name}({input_var}) {{', 'function_def_regex': '(?P<flag_type>.+?)\\s*{function_name}\\s*\$\\s*(?P<inputs>(?:(?:const\\s+[^{{\\&]+\\s+\\&[^{{\\&]+)|(?:[^{{\\&]+))(?:\\s*,\\s*(?:const\\s+[^{{\\&]+\\s+\\&[^{{\\&]+)|(?:[^{{\\&]+))*?)(?:,\\s*(?P<outputs>(?:\\s*(?:[^\\s\\&]+)(?:(?:\\&\\s+)|(?:\\s+(?:\\()?\\&))(?:[^{{])+)+))?\$\\s*\\{{(?P<body>(?:.*?\\n?)*?)(?:(?:return +(?P<flag_var>.+?)?;(?:.*?\\n?)*?\\}})|(?:\\}}))', 'if_begin': 'if ({cond}) {{', 'if_elif': '}} else if ({cond}) {{', 'if_else': '}} else {{', 'import': '#include "{filename}"', 'indent': ' ', 'index': '{variable}[{index}]', 'init_class': 'init_python()', 'init_function': 'init_python()', 'init_instance': 'init_generic()', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_from_schema': 'create_dtype_from_schema("{schema}", {use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray_arr("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': 'YggInput {channel}("{channel_name}", {channel_type});', 'inputs_def_regex': '\\s*(?:const\\s+)?(?P<native_type>(?:[^\\s\\&\\<\\*])+(?:\\<(?P<subtypes>\\s*.+?(?:\\s*,\\s*.+?)*\\s*)\\>)?(\\s+)?(?P<ptr>\\*+)?)(?:\\s*\\&)?(?(ptr)(?(3)(?:\\s*)|(?:\\s+)))(\$(?P<name_ptr>\\*+)?)?(?P<name>[^\\&\\>\\*]+?)(?(5)(?:\$))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'interface': '#include "{interface_library}"', 'line_end': ';', 'not': '!', 'not_flag_cond': '{flag_var} < 0', 'null': 'NULL', 'output': 'YggOutput {channel}("{channel_name}", {channel_type});', 'outputs_def_regex': '\\s*(?P<native_type>(?:[^\\s])+)(\\s+)?(\$)?(?P<ref>\\&)(?(1)(?:\\s*)|(?:\\s+))(?P<name>.+?)(?(2)(?:\$|(?:)))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'print': 'std::cout << "{message}" << std::endl;', 'print_any': 'std::cout << document2string({object}) << std::endl;', 'print_array': 'std::cout << document2string({object}) << std::endl;', 'print_bytes': 'printf("%s\\n", {object});', 'print_class': 'display_python({object});', 'print_complex': 'print_complex({object});', 'print_float': 'printf("%f\\n", {object});', 'print_function': 'display_python({object});', 'print_instance': 'display_generic({object});', 'print_int': 'printf("%i\\n", {object});', 'print_object': 'std::cout << document2string({object}) << std::endl;', 'print_schema': 'std::cout << document2string({object}) << std::endl;', 'print_string': 'printf("%s\\n", {object});', 'print_uint': 'printf("%u\\n", {object});', 'print_unicode': 'printf("%s\\n", {object});', 'python_init': 'init_python_API();', 'quote': '"', 'recv_function': '{channel}.recvRealloc', 'recv_heap': '{channel}.recvRealloc', 'recv_stack': '{channel}.recv', 'return': 'return {output_var};', 'send_function': '{channel}.send', 'true': 'true', 'true_flag': '1', 'try_begin': 'try {', 'try_error_type': 'const std::exception&', 'try_except': '}} catch ({error_type} {error_var}) {{', 'vector_regex': '(?:std\\:\\:)?vector\\<\\s*(?P<type>.*)\\s*\\>', 'while_begin': 'while ({cond}) {{'}¶

classmethod get_json_type(native_type)[source]¶

Get the JSON type from the native language type.

Parameters:: native_type (str) – The native language type.
Returns:: The JSON type.
Return type:: str, dict

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

include_arg_count = True¶

include_channel_obj = False¶

interface_library = 'ygg++'¶

interface_map = {'call': 'flag = {channel_obj}.call({nargs}, {outputs}, {input_refs})', 'client': 'YggRpcClient {channel_obj}("{channel_name}", {datatype_out}, {datatype_in})', 'import': '#include "YggInterface.hpp"', 'input': 'YggInput {channel_obj}("{channel_name}")', 'output': 'YggOutput {channel_obj}("{channel_name}")', 'recv': 'flag = {channel_obj}.recv({nargs}, {input_refs})', 'send': 'flag = {channel_obj}.send({nargs}, {outputs})', 'server': 'YggRpcServer {channel_obj}("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'YggTimesync {channel_obj}("{channel_name}", "{time_units}")'}¶

internal_libraries = {'datatypes': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/datatypes', 'external_dependencies': ['rapidjson', 'python', 'numpy', 'm'], 'include_dirs': [], 'internal_dependencies': ['regex'], 'language': 'c++', 'libtype': 'object'}, 'regex': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_posix': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_win32': {'directory': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c++', 'libtype': 'object', 'source': 'regex_win32.cpp'}, 'ygg++': {'compiler_flags': ['-fPIC'], 'external_dependencies': ['rapidjson', 'python', 'numpy', 'm'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/CPP'], 'internal_dependencies': ['regex', 'datatypes'], 'language': 'c++', 'linker_language': 'c++', 'source': '/opt/hostedtoolcache/Python/3.8.17/x64/lib/python3.8/site-packages/yggdrasil/languages/CPP/YggInterface.cpp'}}¶

inverse_type_map = None¶

classmethod is_cpp_class(var)[source]¶

Determine if a variable uses a C++ class.

Parameters:: var (dict) – Variable.
Returns:: True if it is a C++ class, False otherwise.
Return type:: bool

classmethod is_std_class(var)[source]¶

Determine if a variable utilizing a C++ stdlib class.

Parameters:: var (dict) – Variable.
Returns:: True if var is a C++ stdlib class, False otherwise.
Return type:: bool

language = 'c++'¶

language_aliases = ['cpp', 'cxx']¶

language_ext = ['.cpp', '.CPP', '.cxx', '.C', '.c++', '.cc', '.cp', '.tcc', '.hpp', '.HPP', '.hxx', '.H', '.h++', '.hh', '.hp', '.h']¶

classmethod output2input(var, in_definition=True)[source]¶

Perform conversion necessary to turn an output variable into an corresponding input that can be used to format a function definition.

Parameters:

var (dict) – Variable definition.
in_definition (bool, optional) – If True, the returned dictionary corresponds to an input variable in a function definition. If False, the returned value will correspond to an input to a function. Defaults to True.

Returns:

Updated variable definition.

Return type:

classmethod prepare_input_variables(vars_list, in_definition=False, for_yggdrasil=False)[source]¶

Concatenate a set of input variables such that it can be passed as a single string to the function_call parameter.

Parameters:

vars_list (list) – List of variable dictionaries containing info (e.g. names) that should be used to prepare a string representing input to a function call.
in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying input variables in a function definition. Defaults to False.
for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns:

Concatentated variables list.

Return type:

classmethod requires_length_var(var)[source]¶

Determine if a variable requires a separate length variable.

Parameters:: var (dict) – Dictionary of variable properties.
Returns:: True if a length variable is required, False otherwise.
Return type:: bool

classmethod set_env_class(**kwargs)[source]¶

Set environment variables that are instance independent.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.
Returns:: Environment variables for the model process.
Return type:: dict

type_map = {'1darray': '*', 'any': 'rapidjson::Document', 'array': 'rapidjson::Document', 'boolean': 'bool', 'bytes': 'string_t', 'class': 'python_class_t', 'comm': 'comm_t*', 'complex': 'complex_X', 'dtype': 'dtype_t*', 'flag': 'int', 'float': 'double', 'function': 'python_function_t', 'instance': 'python_instance_t', 'int': 'intX_t', 'integer': 'int', 'length': 'size_t', 'ndarray': '*', 'null': 'void*', 'number': 'double', 'obj': 'rapidjson::ObjWavefront', 'object': 'rapidjson::Document', 'ply': 'rapidjson::Ply', 'schema': 'rapidjson::Document', 'string': 'char*', 'uint': 'uintX_t', 'unicode': 'unicode_t'}¶

classmethod write_doc2vars(channel, std, var_list)[source]¶

Generate the lines of code required to unpack a document into a list of variables.

Parameters:

channel (str) – Name of variable that the channel that the document was received from is stored in.
std (dict) – Variable information for the received document.
var_list (list) – Variables that the document should be unpacked into.

classmethod write_model_recv(channel, recv_var, **kwargs)[source]¶

Write a model receive call include checking the return flag.

Parameters:

channel (str) – Name of variable that the channel being received from was stored in.
recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines required to carry out a receive call in this language.

Return type:

classmethod write_model_send(channel, send_var, **kwargs)[source]¶

Write a model send call include checking the return flag.

Parameters:

channel (str) – Name of variable that the channel being sent to was stored in.
send_var (dict, list) – Information on one or more variables containing information that will be sent.
flag_var (str, optional) – Name of flag variable that the flag should be stored in. Defaults to ‘flag’,
allow_failure (bool, optional) – If True, the returned lines will call a break if the flag is False. Otherwise, the returned lines will issue an error. Defaults to False.

Returns:

Lines required to carry out a send call in this language.

Return type:

classmethod write_try_except(try_contents, except_contents, error_var='e', error_type=None, **kwargs)[source]¶

Return the lines required to complete a try/except block.

Parameters:

try_contents (list) – Lines of code that should be executed inside the try block.
except_contents (list) – Lines of code that should be executed inside the except block.
error_var (str, optional) – Name of variable where the caught error should be stored. Defaults to ‘e’. If ‘…’, the catch clause will catch all errors, but there will not be a name error.
error_type (str, optional) – Name of error type that should be caught. If not provided, defaults to None and will be set based on the class function_param entry for ‘try_error_type’. If ‘…’, the catch clause will catch all errors and error_var will be ignored.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Lines of code perfoming a try/except block.

classmethod write_vars2doc(channel, var_list, std)[source]¶

Generate the lines of code required to pack a list of variables into a document.

Parameters:

channel (str) – Name of variable that the channel that will be used to send the document is stored in.
var_list (list) – Variables that should be packed into the document.
std (dict) – Variable information for the document that will be generated.

class yggdrasil.drivers.CPPModelDriver.ClangPPCompiler(**kwargs)[source]¶

Bases: CPPCompilerBase, ClangCompiler

Interface class for clang++ compiler.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

default_executable = 'clang++'¶

default_flags = ['-g', '-Wall']¶

default_linker = 'clang++'¶

classmethod get_executable_command(args, skip_flags=False, unused_kwargs=None, **kwargs)[source]¶

Determine the command required to run the tool using the specified arguments and options.

Parameters:

args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.
**kwargs – Additional keyword arguments will be passed to the parent class’s method.

Returns:

Output to stdout from the command execution.

Return type:

classmethod get_flags(skip_standard_flag=False, **kwargs)[source]¶

Get a list of compiler flags.

Parameters:

skip_standard_flag (bool, optional) – If True, the C++ standard flag will not be added. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Compiler flags.

Return type:

is_gnu = False¶

is_linker = False¶

toolname = 'clang++'¶

class yggdrasil.drivers.CPPModelDriver.ClangPPLinker(**kwargs)[source]¶

Bases: ClangLinker

Interface class for clang++ linker (calls to ld).

aliases = []¶

default_executable = 'clang++'¶

default_flags = []¶

is_gnu = False¶

languages = ['c++']¶

library_ext = '.so'¶

toolname = 'clang++'¶

toolset = 'llvm'¶

class yggdrasil.drivers.CPPModelDriver.GPPCompiler(**kwargs)[source]¶

Bases: CPPCompilerBase, GCCCompiler

Interface class for G++ compiler/linker.

aliases = ['gnu-c++']¶

default_executable = 'g++'¶

default_flags = ['-g', '-Wall']¶

default_linker = 'g++'¶

classmethod get_flags(skip_standard_flag=False, **kwargs)[source]¶

Get a list of compiler flags.

Parameters:

skip_standard_flag (bool, optional) – If True, the C++ standard flag will not be added. Defaults to False.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Compiler flags.

Return type:

is_gnu = True¶

is_linker = False¶

toolname = 'g++'¶

class yggdrasil.drivers.CPPModelDriver.GPPLinker(**kwargs)[source]¶

Bases: GCCLinker

Interface class for clang++ linker (calls to ld).

aliases = ['gnu-c++']¶

default_executable = 'g++'¶

default_flags = []¶

is_gnu = True¶

languages = ['c++']¶

library_ext = '.so'¶

toolname = 'g++'¶

toolset = 'gnu'¶

class yggdrasil.drivers.CPPModelDriver.MSVCPPCompiler(**kwargs)[source]¶

Bases: CPPCompilerBase, MSVCCompiler

Inteface class for MSVC compiler when compiling C++.

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

default_archiver = 'LIB'¶

default_executable = 'cl'¶

default_flags = ['/W4', '/Zi', '/FS', '/EHsc', '/bigobj', '/TP', '/nologo', '-D_CRT_SECURE_NO_WARNINGS']¶

default_linker = 'LINK'¶

dont_create_linker = True¶

is_gnu = False¶

search_path_flags = None¶

toolname = 'cl++'¶

yggdrasil.drivers.CompiledModelDriver module¶

class yggdrasil.drivers.CompiledModelDriver.ArchiverBase(**kwargs)[source]¶

Bases: LinkerBase

Base class for archivers.

static_library_flag¶

Flag that should be prepended to the archiver tool arguments to indicated that a static library should be produced.

Type:: str

library_name_key¶

Option key indicating the name of a library that should be linked against.

Type:: str

library_directory_key¶

Option key indicating a directory that should be included in the linker search path for libraries.

Type:: str

library_prefix¶

Prefix that should be added to library paths.

Type:: str

library_ext¶

Extension that should be used for static libraries.

Type:: str

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

flag_options = {}¶

classmethod get_flags(**kwargs)[source]¶

Get a list of flags for this archiver tool.

Parameters:: **kwargs – Additional keyword arguments are passed to the parent class’s method.
Returns:: Archiver flags.
Return type:: list

classmethod get_output_file(obj, **kwargs)[source]¶

Determine the appropriate output file that will result when linking a given object file.

Parameters:

obj (str) – Object file being linked that name base will be taken from.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Full path to file that will be produced.

Return type:

library_ext = None¶

static_library_flag = '-static'¶

tooltype = 'archiver'¶

class yggdrasil.drivers.CompiledModelDriver.BuildToolBase(**kwargs)[source]¶

Bases: CompilerBase

Base class for build tools which are used to coordinate compilation.

Parameters:

buildfile (str, optional) – File containing information about the build. Defaults to default_buildfile class attribute, if set.
builddir (str, optional) – Directory where build files should be placed. Defaults to directory where build is called.
sourcedir (str, optional) – Directory where source files are stored. Defaults to directory where build is called.
target (str, optional) – Build target. If not provided, build will be created without a target.

Class Attributes:

default_buildfile = None¶

flag_options = {}¶

tooltype = 'buildtool'¶

class yggdrasil.drivers.CompiledModelDriver.CompilationToolBase(**kwargs)[source]¶

Bases: object

Base class for compilation command line tools.

Class Attributes:

toolname (str): Tool name used for registration and as a default for the: executable. [REQUIRED]

aliases (list): Alternative names that the tool might have. tooltype (str): Tool type. One of ‘compiler’, ‘linker’, or ‘archiver’.

[AUTOMATED]

languages (list): Programming languages that this tool can be used on.: [REQUIRED]
platforms (list): Platforms that the tool is available on. Defaults to: [‘Windows’, ‘MacOS’, ‘Linux’].
default_executable (str): The default tool executable command if: different than the toolname.
default_executable_env (str): Environment variable where the executable: command might be stored.
default_flags (list): Default flags that should be used when calling the: tool (e.g. for verbose output or enhanced warnings).
default_flags_env (str): Environment variable where default flags for: the tools might be stored.

output_key (str): Option key indicating the output file name. output_first (bool): If True, the output key (and its value) are put in

front of the other flags.

flag_options (collections.OrderedDict): Mappings between keyword: arguments passed to get_flags and command line tool flags. Keys indicate the keyword argument that will be processed in Python and values are the string used with the tool on the command line to indicate the desired flag. Flags that contain ‘%s’ will be formatted with values passed to get_flags as the designated keyword argument. Flags that do not contain ‘%s’ will assumed to act on the following argument. If the value passed to get_flags under the designated keyword argument is not a boolean, it will be appended to the flag list following the corresponding flag. If the value is a boolean and it is True, only the flag will be added to the list of flags. The order of entries indicates the order the flags should be added to the list.
search_path_envvar (list): Environment variables containing a list of paths: to search for library files. Either search_path_envvar or search_path_flags must be set. [REQUIRED]
search_path_env (list): Paths relative to the env prefix that should: be searched if the VIRTUAL_ENV or CONDA_PREFIX environment variable is set.
search_path_flags (list): Flags that should be passed to the tool: executable in order to locate the search path. Either search_path_envvar or search_path_flags must be set. [REQUIRED]
search_regex_begin (str): Search string indicating where the set of: paths begins in the output from running the tool executable with the search_path_flags. If None, the search is performed from the very beginning of the output.
search_regex_end (str): Search string indicating where the set of: paths ends in the output from running the tool executable with the search_path_flags. If None, the search is performed to the very end of the output.
search_regex (list): Regex strings that should be used to locate paths: in the output from running the tool executable with the search_path_flags.
product_exts (list): List of extensions that will be added to the: output file’s base to get a list of products that might be produced by calling the compilation tool.
product_files (list): List of file basenames that will be joined with: the directory containing each output to get a list of products that might be produced by calling the compilation tool.
remove_product_exts (list): List of extensions or directories matching: entries in product_exts and product_files that should be removed during cleanup. Be careful when adding files to this list.

asan_flags (list): Flags added when with_asan is specified.

aliases = []¶

classmethod append_flags(out, key, value, **kwargs)[source]¶

Append one ore more flag(s) to a list of flags based on a key value/set.

Parameters:

out (list) – Existing list that the new flag should be appended to.
key (str) – String identifying the type of flag. See create_flag for additional details.
value (object) – Value that should be set in the flag. If a list, multiple flags are added, one for each item in the list.
prepend (bool, optional) – If True, new flags are prepended to the front of the list rather than the end. Defaults to False. This keyword argument is ignoerd if position is provided.
position (int, optional) – If not None, this is the position that the new elements should be added to the existing flags. Positive positions indicate indexes into the existing list of flags. Negative are relative to the end of the existing list such that -1 is the same as just appending the new flags to the end of the list. Defaults to None and prepend takes precedence.
no_duplicates (bool, optional) – If True, the new flags being added will be checked against the existing ones to ensure that there are not duplicates. If False, the new flags are added reguardless of the existing flags. Defaults to False.
allow_duplicate_values (bool, optional) – If True, the same key can be added with the same value more than once. Otherwise, only the first instance of the value is added. Defaults to False.

Raises:

ValueError – If there are unexpected keyword arguments.
ValueError – If no_duplicates is True and the existing list of flags already contains a flag matching the provided flag key.

classmethod append_product(products, src, new, new_dir=None, dont_append_src=False)[source]¶

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters:

products (list) – List of of existing products that new product should be appended to.
src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).
new (str) – New product that should be appended to the list.
new_dir (str, optional) – Directory that should be used as base when adding files listed in cls.product_files. Defaults to os.path.dirname(new).
dont_append_src (bool, optional) – If True and src is in the list of products, it will be removed. Defaults to False.

asan_flags = None¶

static before_registration(cls)[source]¶: Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

classmethod call(args, language=None, toolname=None, skip_flags=False, dry_run=False, out=None, overwrite=False, products=None, allow_error=False, working_dir=None, additional_args=None, suffix='', cache_key=None, **kwargs)[source]¶

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters:

args (list) – The arguments that should be passed to the tool.
language (str, optional) – Language of tool that should be used. If different than the languages supported by the current tool, the correct tool is used instead. Defaults to None and is ignored.
toolname (str, optional) – Name of compilation tool that should be used. Defaults to None and the default tool for the language will be used.
skip_flags (bool, optional) – If True, args is assumed to include any necessary flags. If False, args are assumed to the files that the tool is called on and flags are determined from them. Defaults to False.
dry_run (bool, optional) – If True, the tool won’t be called, but the products will be updated. Defautls to False.
out (str, optional) – Full path to output file that should be created. If None, the path will be determined from the path to the first argument provided. Defaults to None. This keyword argument will be ignored if skip_flags is True.
overwrite (bool, optional) – If True, the existing compile file will be overwritten. Otherwise, it will be kept and this function will return without recompiling the source file.
products (list, optional) – Existing Python list that additional products produced by the compilation should be appended to. Defaults to None and is ignored.
allow_error (bool, optional) – If True and there is an error when call the executable, it will be ignored. If False, errors will result in an exception being raised. Defaults to False.
working_dir (str, optional) – Working directory where tool should be called from. This will also be used to construct the path for the output file. Defaults to None and is ignored.
additional_args (list, optional) – Additional arguments that should be appended to args before continuing. Defaults to None and is ignored.
suffix (str, optional) – Suffix that should be added to the output file (before the extension). Defaults to “”.
cache_key (str, optional) – Key that should be used to cache results so that they may be used multiple times. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are passed to cls.get_executable_command. and tools.popen_nobuffer.

Returns:

Output to stdout from the command execution if skip_flags is: True, produced file otherwise.

Return type:

Raises:

RuntimeError – If there is an error when running the command and allow_error is False.

compatible_toolsets = []¶

classmethod create_flag(key, value)[source]¶

Create a flag from a key/value set.

Parameters:

key (str) – String identifying the type of flag. If key contains ‘%s’, it is assumed that the flag will be produced by formatting the value according to key. If key dosn’t contain ‘%s’, it is assumed that key indicates that value is the following item and they both will be returned. If key is an empty string, then the value is assumed to constitute the entire flag.
value (object) – Value that should be set in the flag. If a list, multiple flags are returned, one for each item in the list.

Returns:

Items representing the flag.

Return type:

default_executable = None¶

default_executable_env = None¶

default_flags = []¶

default_flags_env = None¶

classmethod env_matches_tool(use_sysconfig=False, env=None, with_flags=False)[source]¶

Determine if the executable pointed to by any environment variable matches this compilation tool.

Parameters:

use_sysconfig (bool, optional) – If True, check the sysconfig variables, otherwise check os.environ. Defaults to False.
env (dict, optional) – Existing dictionary that should be updated with variables. Defaults to None and is ignored.
with_flags (bool, optional) – If True, preserve any flags included in the environment variable. Defaults to False.

Returns:

True if the environment variable matches, False otherwise.

Return type:

bool

classmethod file2base(fname)[source]¶

Determine basename from path.

Parameters:: fname (str) – Full or partial path to file.
Returns:: File name without extension.
Return type:: str

flag_options = {}¶

classmethod get_alternate_class(toolname=None, language=None)[source]¶

Return an alternate class to use if the provided toolname dosn’t match the current tool.

Parameters:

toolname (str, optional) – Name of compilation tool that should be used. Defaults to None and the current toolname is assumed.
language (str, optional) – Language that alternate class should support. Defaults to None and the current language will be assumed.

Returns:

The compilation tool that corresponds: to the provided toolname.

Return type:

classmethod get_env_flags()[source]¶

Get a list of flags stored in the environment variables.

Returns:: Flags for the tool.
Return type:: list

classmethod get_env_prefixes()[source]¶

Determine the virtualenv/conda path prefixes.

Returns:

Virtualenv/conda path prefixes. Empty list will be: returned if virtualenv/conda are not active.

Return type:

classmethod get_executable(full_path=False)[source]¶

Determine the executable that should be used to call this tool.

Parameters:: full_path (bool, optional) – If True the full path to the executable file will be returned. Defaults to False.
Returns:: Name of (or path to) the tool executable.
Return type:: str

classmethod get_executable_command(args, skip_flags=False, unused_kwargs=None, use_ccache=False, **kwargs)[source]¶

Determine the command required to run the tool using the specified arguments and options.

Parameters:

args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.
skip_flags (bool, optional) – If True, args is assumed to include any necessary flags. If False, args are assumed to the files that the tool is called on and flags are determined from them. Defaults to False.
unused_kwargs (dict, optional) – Existing Python dictionary that unused keyword arguments will be added to. Defaults to None and is initialized to an empty dict.
use_ccache (bool, optional) – If True, ccache will be added to the compilation executable. Defaults to False.
**kwargs – Additional keyword arguments are ignored and stored in unused_kwargs if provided.

Returns:

Output to stdout from the command execution.

Return type:

classmethod get_flags(flags=None, outfile=None, output_first=None, unused_kwargs=None, skip_defaults=False, dont_skip_env_defaults=False, with_asan=False, **kwargs)[source]¶

Get a list of flags for the tool.

Parameters:

flags (list, optional) – User defined flags that should be included. Defaults to empty list.
outfile (str, optional) – If provided, it is appended to the end of the flags following the cls.output_key flag to indicate that this is the name of the output file. Defaults to None and is ignored.
output_first (bool, optional) – If True, output flag(s) will be placed at the front of the returned flags. If False, they are placed at the end. Defaults to None and is set by cls.output_first.
unused_kwargs (dict, optional) – Existing dictionary that unused keyword arguments should be added to. Defaults to None and is ignored.
skip_defaults (bool, optional) – If True, the default flags will not be added. Defaults to False.
dont_skip_env_defaults (bool, optional) – If skip_defaults is True, and this keyword is True, the flags from the environment variable will be added. Defaults to False.
remove_flags (list, optional) – List of flags to remove. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are ignored and added to unused_kwargs if provided.

Returns:

Flags for the tool.

Return type:

classmethod get_language_ext()[source]¶

Get the extensions associated with the language that this tool can handle.

Returns:: Language file extensions.
Return type:: list

classmethod get_search_path(env_only=False, libtype=None, cfg=None)[source]¶

Determine the paths searched by the tool for external library files.

Parameters:

env_only (bool, optional) – If True, only the search paths as indicated by a virtualenv/conda environment are returned. Defaults to False.
libtype (str, optional) – Library type being searched for. Defaults to None.
cfg (YggConfigParser, optional) – Configuration object currently being updated. Defaults to the global configuration.

Returns:

List of paths that the tools will search.

Return type:

classmethod get_source_products(products, source_products=[])[source]¶

Get the list of products that should be removed without checking for source files based on cls.remove_product_exts.

Parameters:

products (list) – List of products that should be checked against cls.remove_product_exts.
source_products (list, optional) – Existing list of products that new source_products should be added to. Defaults to [].

Returns:

Products that should be removed without checking for source: files.

Return type:

classmethod get_tool_suffix()[source]¶

Get the string that should be added to tool products based on the tool used.

Returns:

Suffix that should be added to tool products to indicate the: tool used.

Return type:

is_build_tool = False¶

is_gnu = False¶

classmethod is_installed()[source]¶

Determine if this tool is installed by looking for the executable.

Returns:: True if the tool is installed, False otherwise.
Return type:: bool

languages = []¶

classmethod locate_file(fname, libtype=None, verbose=False, **kwargs)[source]¶

Locate a library file.

Parameters:

fname (str) – Name of library.
libtype (str, optional) – Library type being searched for. Defaults to None.
verbose (bool, optional) – If True, display information about the success or failure of the search. Defaults to False.
**kwargs – Additional keyword arguments are passed to get_search_path.

Returns:

Full path to located library file.

Return type:

object_tool = None¶

output_first = False¶

output_key = '-o'¶

platforms = ['Windows', 'MacOS', 'Linux']¶

product_exts = []¶

product_files = []¶

remove_product_exts = []¶

classmethod remove_products(src, new)[source]¶

Remove products produced during compilation.

Parameters:

src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).
new (str) – The full path to the primary product of the compilation call (usually an object, executable, or library).

search_path_env = None¶

search_path_envvar = None¶

search_path_flags = None¶

search_regex = ['([^\n]+)']¶

search_regex_begin = None¶

search_regex_end = None¶

classmethod set_env(existing=None, **kwargs)[source]¶

Set environment variables required for compilation.

Parameters:

existing (dict, optional) – Existing dictionary of environment variables that new variables should be added to. Defaults to a copy of os.environ.
**kwargs – Additional keyword arguments are ignored.

Returns:

Environment variables for the model process.

Return type:

source_product_exts = []¶

tool_suffix_format = '_%sx'¶

classmethod tool_version(**kwargs)[source]¶

Get the version of the compilation tool.

Returns:: Version.
Return type:: str

toolname = None¶

toolset = None¶

tooltype = None¶

version_flags = ['--version']¶

classmethod write_wrappers(**kwargs)[source]¶

Write any wrappers needed to compile and/or run a model.

Parameters:: **kwargs – Keyword arguments are ignored (only included to allow cascade from child classes).
Returns:: Full paths to any created wrappers.
Return type:: list

class yggdrasil.drivers.CompiledModelDriver.CompilationToolMeta(name, bases, class_dict)[source]¶

Bases: type

Meta class for registering compilers.

class yggdrasil.drivers.CompiledModelDriver.CompiledModelDriver(*args, **kwargs)[source]¶

Bases: ModelDriver

Base class for models written in compiled languages.

Parameters:

name (str) – Driver name.
args (str or list) – The model executable and any arguments that should be passed to the model executable.
source_files (list, optional) – Source files that should be compiled into an executable. Defaults to an empty list and the driver will search for a source file based on the model executable (the first model argument).
compiler (str, optional) – Command or path to executable that should be used to compile the model. If not provided, the compiler will be determined based on configuration options for the language (if present) and the registered compilers that are available on the current operating system.
compiler_flags (list, optional) – Flags that should be passed to the compiler during compilation. If nto provided, the compiler flags will be determined based on configuration options for the language (if present), the compiler defaults, and the default_compiler_flags class attribute.
linker (str, optional) – Command or path to executable that should be used to link the model. If not provided, the linker will be determined based on configuration options for the language (if present) and the registered linkers that are available on the current operating system
linker_flags (list, optional) – Flags that should be passed to the linker during compilation. If not provided, the linker flags will be determined based on configuration options for the language (if present), the linker defaults, and the default_linker_flags class attribute.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).
**kwargs – Additional keyword arguments are passed to parent class.

Class Attributes:

default_compiler (str): Name of compiler that will be used if not: set explictly by instance or config file.
default_compiler_flags (list): Flags that should be passed to the: compiler by default for this language.
default_linker (str): Name of linker that will be used if not: set explictly by instance or config file
default_linker_flags (list): Flags that should be passed to the: linker by default for this language.
allow_parallel_build (bool): If True, a file can be compiled by: two processes simultaneously. If False, it cannot and an MPI barrier will be used to prevent simultaneous compilation. Defaults to False.

source_files¶

Source files.

Type:: list

compiler¶

Name or path to the compiler that should be used.

Type:: str

compiler_flags¶

Compiler flags.

Type:: list

linker¶

Name or path to the linker that should be used.

Type:: str

linker_flags¶

Linker flags.

Type:: list

compiler_tool¶

Compiler tool that will be used.

Type:: CompilerBase

linker_tool¶

Linker tool that will be used.

Type:: LinkerBase

static after_registration(cls, **kwargs)[source]¶: Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

allow_parallel_build = False¶

buildfile_locked(dry_run=False)[source]¶: Context manager for locked build file.

classmethod call_compiler(src, language=None, toolname=None, dont_build=None, **kwargs)[source]¶

Compile a source file into an executable or linkable object file, checking for errors.

Parameters:

src (str) – Full path to source file.
out (str, optional) – Full path to the output object file that should be created. Defaults to None and is created from the provided source file.
flags (list, optional) – Compilation flags. Defaults to []. If compiler_flags is present, flags is replaced by compiler_flags.
compiler_flags (list, optional) – Alternative to flags. Ignored if not provided.
dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.
dont_build (bool, optional) – If True, cmake configuration/generation will be run, but the project will not be built. Defaults to None. If provided, this overrides dont_link.
overwrite (bool, optional) – If True, the existing compile file will be overwritten. Otherwise, it will be kept and this function will return without recompiling the source file.
language (str, optional) – Language that should be used to compile the files. Defaults to None and the language of the current driver is used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
products (list, optional) – Existing Python list that additional products produced by the compilation should be appended to. Defaults to None and is ignored.
**kwargs – Additional keyword arguments are passed to run_executable. and call_linker if dont_link is False.

Returns:

Full path to compiled source.

Return type:

Raises:

RuntimeError – If there is an error in calling the compiler.
RuntimeError – If the compilation command dosn’t yield the specified output file.

classmethod call_linker(obj, language=None, toolname=None, **kwargs)[source]¶

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters:

obj (list) – Object files that should be linked.
language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to run_executable.

Returns:

Full path to compiled source.

Return type:

classmethod cleanup_dependencies(products=None, verbose=False, **kwargs)[source]¶: Cleanup dependencies.

classmethod compile_dependencies(toolname=None, dep=None, **kwargs)[source]¶: Compile any required internal libraries, including the interface.

compile_dependencies_instance(*args, **kwargs)[source]¶: Compile dependencies specifically for this instance.

compile_model(source_files=None, skip_interface_flags=False, **kwargs)[source]¶

Compile model executable(s).

Parameters:

source_files (list, optional) – Source files that will be compiled. Defaults to None and is set to the source_files attribute.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. This includes the logger flag specifying the current logging level. Defaults to False.
**kwargs – Keyword arguments are passed on to the call_compiler method.

Returns:

Compiled model file path.

Return type:

classmethod configuration_steps()[source]¶

Get a list of configuration steps with tuples of flags and boolean values.

Returns:

Pairs of descriptions and states for: different steps in the configuration all steps must be True for the language to be configured.

Return type:

OrderedDict

classmethod configure(cfg, **kwargs)[source]¶

Add configuration options for this language.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.
**kwargs – Additional keyword arguments are used to set tool configuration options (e.g. ‘compiler’).

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_executable_type(cfg)[source]¶

Add configuration options specific in the executable type before the libraries are configured.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_libraries(cfg)[source]¶

Add configuration options for external libraries in this language.

Parameters:

cfg (CisConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

classmethod configure_library(cfg, k)[source]¶

Add configuration options for an external library.

Parameters:

cfg (YggConfigParser) – Config class that options should be set for.

Returns:

Section, option, description tuples for options that could not: be set.

Return type:

default_archiver = None¶

default_archiver_flags = None¶

default_compiler = None¶

default_compiler_flags = None¶

default_linker = None¶

default_linker_flags = None¶

classmethod executable_command(args, exec_type='compiler', toolname=None, **kwargs)[source]¶

Compose a command for running a program using the compiler for this language and the provied arguments. If not already present, the compiler command and compiler flags are prepended to the provided arguments.

Parameters:

args (list) – The program that returned command should run and any arguments that should be provided to it. For the compiler, this means the source files, for the linker, this means the object files.
exec_type (str, optional) – Type of executable command that will be returned. If ‘compiler’, a command using the compiler is returned, if ‘linker’, a command using the linker is returned, and if ‘direct’, the raw args being provided are returned. Defaults to ‘compiler’.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to either get_linker_flags or get_compiler_flags.

Returns:

Arguments composing the command required to run the program: from the command line using the compiler for this language.

Return type:

Raises:

ValueError – If exec_type is not ‘compiler’, ‘linker’, or ‘direct’.

executable_type = 'compiler'¶

external_libraries = {}¶

classmethod get_available_tools(tooltype)[source]¶

Return the registry of compilation tools for this language.

Parameters:: tooltype (str) – Name of tool type that should be returned. Values include ‘compiler’, ‘linker’, or ‘archiver’.
Returns:: Registry of tools for this language.
Return type:: dict

classmethod get_buildfile_lock(fname=None, context=None, instance=None, **kwargs)[source]¶

Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

Parameters:

name (str) – Build file.
context (threading.Context) – Threading context.
instance (ModelDriver) – Driver instance that should be used.
**kwargs – Additional keyword arguments are passed to the FileLock initialization.

Returns:

Lock for the buildfile.

Return type:

FileLock

classmethod get_compiler_flags(toolname=None, compiler=None, **kwargs)[source]¶

Determine the flags required by the current compiler.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
compiler (CompilerBase, optional) – Compilation tool class for the compiler that should be used. Defaults to None and is set based on toolname.
**kwargs – Keyword arguments are passed to cls.update_compiler_kwargs first and then the compiler’s get_flags method.

Returns:

Flags for the compiler.

Return type:

classmethod get_dependency_include_dirs(dep, toolname=None, default=None)[source]¶

Get the include directories for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns:

Full paths to the directories containing the dependency’s: header(s).

Return type:

Raises:

ValueError – If the include directory cannot be determined for the specified dependency and default is None.

classmethod get_dependency_info(dep, toolname=None, default=None)[source]¶

Get the dictionary of information associated with a dependency.

Parameters:

dep (str) – Name of internal or external dependency.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (dict, optional) – Information dictionary that should be returned if dep cannot be located. Defaults to None and an error will be raised if dep cannot be found.

Returns:

Dependency info.

Return type:

classmethod get_dependency_library(dep, default=None, libtype=None, commtype=None, toolname=None, disable_python_c_api=False, with_asan=False)[source]¶

Get the library location for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.
libtype (str, optional) – Library type that should be returned. Valid values are ‘static’ and ‘shared’. Defaults to None and will be set on the dependency’s libtype (if it has one) or the _default_libtype parameter (if it doesn’t).
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Full path to the library file. For header only libraries,: an empty string will be returned.

Return type:

Raises:

ValueError – If libtype is not ‘static’ or ‘shared’.
ValueError – If the path to the library cannot be determined for the specified dependency and default is None.

classmethod get_dependency_object(dep, default=None, commtype=None, toolname=None, disable_python_c_api=False, with_asan=False)[source]¶

Get the location of an object file for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the object file.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used in the names of internal libraries.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Full path to the object file.

Return type:

classmethod get_dependency_order(deps, toolname=None, disable_python_c_api=False)[source]¶

Get the correct dependency order, including any dependencies for the direct dependencies.

Parameters:

deps (list) – Dependencies in order.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.

Returns:

Dependency order.

Return type:

classmethod get_dependency_source(dep, toolname=None, default=None)[source]¶

Get the path to the library source files (or header files) for a dependency.

Parameters:

dep (str) – Name of internal or external dependency or full path to the library.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
default (str, optional) – Default that should be used if a value cannot be determined form internal/external dependencies or if dep is not a valid file. Defaults to None and is ignored.

Returns:

Full path to the library source file. For header only libraries: this will be the header location.

Return type:

classmethod get_external_libraries(no_comm_libs=False)[source]¶

Determine the external libraries that are required based on the default comm.

Parameters:

no_comm_libs (bool, optional) – If True, libraries for the installed comms are not included in the returned list. Defaults to False.

Returns:

The names of external libraries required by the interface: library, including the dependency libraries for the installed comm libraries.

Return type:

classmethod get_internal_suffix(commtype=None, disable_python_c_api=False, with_asan=False)[source]¶

Determine the suffix that should be used for internal libraries.

Parameters:

commtype (str, optional) – If provided, this is the communication type that should be used for the model. If None, the default comm is used.
disable_python_c_api (bool, optional) – If True, the Python C API will be disabled. Defaults to False.
with_asan (bool, optional) – If True, the model will be compiled and linked with the address sanitizer enabled (if there is one available for the selected compiler).

Returns:

Suffix that should be added to internal libraries to: differentiate between different dependencies.

Return type:

classmethod get_linker_flags(toolname=None, **kwargs)[source]¶

Determine the flags required by the current linker.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Keyword arguments are passed to cls.update_linker_kwargs first and then the linker’s get_flags method.

Returns:

Flags for the linker.

Return type:

classmethod get_testing_options(**kwargs)[source]¶

Method to return a dictionary of testing options for this class.

Parameters:

**kwargs – Additional keyword arguments are passed to the parent class.

Returns:

Dictionary of variables to use for testing. Key/value pairs:: kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type:

classmethod get_tool(*args, **kwargs)[source]¶

Get tool from a driver class.

Parameters:

*args – Arguments are passed to the get_tool_static method.
**kwargs – Keyword arguments are passed to the get_tool_static method.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

get_tool_instance(*args, **kwargs)[source]¶

Get tool from a driver instance.

Parameters:

*args – Arguments are passed to the get_tool_static method.
**kwargs – Keyword arguments are passed to the get_tool_static method.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

static get_tool_static(cls, tooltype, toolname=None, return_prop='tool', default=False, language=None)[source]¶

Get the class associated with the specified compilation tool for this language.

Parameters:

cls (class, instance) – Compiled driver class or instance of compiled driver class to get tool for.
tooltype (str) – Type of compilation tool that should be returned.
toolname (str, optional) – Name of the tool that should be returned. Defaults to None and the tool name associated with the provided class/instance will be used.
return_prop (str, optional) – Value that should be returned. If ‘tool’, the tool is returned. If ‘name’, the tool name is returned. If ‘flags’, the tool flags are returned. Defaults to ‘tool’.
default (object, optiona) – Tool that should be returned if one cannot be identified. If False, an error will be raised when a tool cannot be located. Defaults to False.
language (str, optional) – Language of tools that should be returned. Defaults to None if not provided.

Returns:

Class providing an interface to the specified: compilation tool.

Return type:

Raises:

NotImplementedError – If a tool is not specified.
ValueError – If return_prop is not ‘tool’, ‘name’, or ‘flags’.

classmethod identify_source_files(args=None, working_dir=None, source_files=None, **kwargs)[source]¶

Determine the source file based on model arguments.

Parameters:

args (list, optional) – Arguments provided.
working_dir (str, optional) – Working directory.
source_files (list, optional) – Source files in the model.
**kwargs – Additional keyword arguments are ignored.

Returns:

Source files.

Return type:

internal_libraries = {}¶

is_build_tool = False¶

classmethod is_language_installed()[source]¶

Determine if the interpreter/compiler for the associated programming language is installed.

Returns:: True if the language interpreter/compiler is installed.
Return type:: bool

classmethod is_library_installed(lib, cfg=None)[source]¶

Determine if a dependency is installed by check for the appropriate config options setting the path to the library files.

Parameters:

lib (str) – Name of the library that should be checked.
cfg (CisConfigParser, optional) – Config class that should be checked. Defaults to yggdrasil.config.ygg_cfg if not provided.

Returns:

True if the library is installed, False otherwise.

Return type:

bool

classmethod is_standard_library(dep)[source]¶

Determine if a dependency is a standard library.

Parameters:

dep (str, tuple) – Dependency name or tuple with language and dependency name.

Returns:

True if the dependency is a standard library, false: otherwise.

Return type:

bool

classmethod is_tool_installed(tooltype)[source]¶

Determine if a compilation tool of a certain is installed for this language.

Parameters:: tooltype (str) – Type of tool to check for. Supported values include ‘compiler’, ‘linker’, & ‘archiver’.
Returns:: True if a tool of the specified type is installed.
Return type:: bool

kwargs_in_suffix = ['with_asan', 'disable_python_c_api', 'commtype']¶

classmethod language_executable(toolname=None)[source]¶

Command required to compile/run a model written in this language from the command line.

Parameters:

toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns:

Name of (or path to) compiler/interpreter executable required: to run the compiler/interpreter from the command line.

Return type:

classmethod language_version(toolname=None, **kwargs)[source]¶

Determine the version of this language.

Parameters:

**kwargs – Keyword arguments are passed to cls.run_executable.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

Returns:

Version of compiler/interpreter for this language.

Return type:

locked_buildfile = None¶

model_command()[source]¶

Return the command that should be used to run the model.

Returns:

Any commands/arguments needed to run the model from the: command line.

Return type:

classmethod mpi_partner_cleanup(self)[source]¶: Actions cleaning up an MPIPartnerModel.

classmethod mpi_partner_init(self)[source]¶: Actions initializing an MPIPartnerModel.

parse_arguments(args, **kwargs)[source]¶

Sort model arguments to determine which one is the executable and which ones are arguments.

Parameters:

args (list) – List of arguments provided.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod partner_buildfile_lock(self)[source]¶: Actions completing buildfile lock on MPIPartnerModels.

run_model(**kwargs)[source]¶

Run the model. Unless overridden, the model will be run using run_executable.

Parameters:: **kwargs – Keyword arguments are passed to run_executable.

classmethod select_suffix_kwargs(kwargs)[source]¶

Select suffix kwargs.

Parameters:: kwargs (dict) – Keyword arguments to select suffix kwargs from.
Returns:: Selected suffix kwargs.
Return type:: dict

set_env(for_compile=False, compile_kwargs=None, toolname=None, **kwargs)[source]¶

Get environment variables that should be set for the model process.

Parameters:

for_compile (bool, optional) – If True, environment variables are set that are necessary for compiling. Defaults to False.
compile_kwargs (dict, optional) – Keyword arguments that should be passed to the compiler’s set_env method. Defaults to empty dict.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

classmethod set_env_compiler(compiler=None, toolname=None, **kwargs)[source]¶

Get environment variables that should be set for the compilation process.

Parameters:

compiler (CompilerBase, optional) – Compiler that set_env shoudl be called for. If not provided, the default compiler for this language will be used.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns:

Environment variables for the model process.

Return type:

set_target_language()[source]¶

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns:: Name of language.
Return type:: str

standard_libraries = []¶

classmethod update_compiler_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, directory=None, include_dirs=None, definitions=None, skip_interface_flags=False, **kwargs)[source]¶

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters:

dont_link (bool, optional) – If True, the command will result in a linkable object file rather than an executable. Defaults to False.
for_api (bool, optional) – If True, flags are added that are required for compiling internal api libraries in this language. This includes external communication libraries. Defaults to False.
for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
include_dirs (list, optional) – If provided, each list element will be added as an included directory flag. Defaults to None and is initialized as an empty list.
definitions (list, optional) – If provided, each list element will be added as a defintion. Defaults to None and is initialized to an empty list.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.
internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.
external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.
**kwargs – Additional keyword arguments are passed to the compiler class’s ‘get_flags’ method and get_linker_flags if dont_link is False.

Returns:

Keyword arguments for a get_flags method providing compiler: flags.

Return type:

classmethod update_linker_kwargs(for_api=False, for_model=False, commtype=None, toolname=None, libtype='object', skip_interface_flags=False, use_library_path_internal=False, **kwargs)[source]¶

Update keyword arguments supplied to the linker/archiver get_flags method for various options.

Parameters:

for_api (bool, optional) – If True, flags are added that are required for linking internal api libraries in this language. This includes external communication libraries. Defaults to False.
for_model (bool, optional) – If True, flags are added that are required for including the interface library. Defaults to False.
commtype (str, optional) – If provided, this is the communication type that should be used for the model and flags for just that comm type will be included. If None, flags for all installed comm types will be included. Default to None. This keyword is only used if for_model is True.
toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.
libtype (str, optional) – Library type that should be created by the linker/archiver. Valid values are ‘static’, ‘shared’, or ‘object’. Defaults to ‘object’.
skip_interface_flags (bool, optional) – If True, interface flags will not be added. Defaults to False.
libraries (list, optional) – Full paths to libraries that should be linked against. Defaults to an empty list.
internal_dependencies (list, optional) – If provided, a list of names of internal libraries that are required or linkable object files for dependencies. Defaults to an empty list.
external_dependencies (list, optional) – If provided, a list of names of external libraries that are required or linkable object files for dependencies. Defaults to an empty list.
use_library_path_internal (bool, optional) – If True, internal dependencies are included as full paths. Defaults to False.
**kwargs – Additional keyword arguments are passed to the linker (or archiver if static is True) ‘get_flags’ method.

Returns:

Keyword arguments for a get_flags method providing linker/: archiver flags.

Return type: