yggdrasil.languages.Python package

Submodules

yggdrasil.languages.Python.CisInterface module

yggdrasil.languages.Python.YggInterface module

yggdrasil.languages.Python.YggInterface.InterfaceComm(name, comm_class=None, global_scope=False, **kwargs)[source]

Short hand for initializing a default comm for use as an interface.

Parameters
  • name (str) – The name of the message queue.

  • comm_class (CommBase, optional) – Communication class that should be used. Defaults to DefaultComm if not provided.

  • global_scope (bool, optional) – If True, the comm will be checked for in the global scope before creating a new one. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to DefaultComm.

yggdrasil.languages.Python.YggInterface.YggArrayInput(name, **kwargs)[source]

Get class for handling table-like formatted input as arrays.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggArrayOutput(name, **kwargs)[source]

Get class for handling table-like formatted output as arrays.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiArrayInput(name, **kwargs)[source]

Get class for handling table-like formatted input as arrays.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggAsciiTableInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiArrayOutput(name, fmt=None, **kwargs)[source]

Get class for handling table-like formatted output as arrays.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • fmt (str) – A C style format string specifying how each ‘row’ of output should be formated. This should include the newline character.

  • **kwargs – Additional keyword arguments are passed to YggAsciiTableOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiFileInput(name, **kwargs)[source]

Get class for generic ASCII input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiFileOutput(name, **kwargs)[source]

Get class for generic ASCII output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiTableInput(name, as_array=False, **kwargs)[source]

Get class for handling table-like formatted input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • as_array (bool, optional) – If True, recv returns the entire table array and can only be called once. If False, recv returns row entries. Default to False.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggAsciiTableOutput(name, fmt=None, as_array=False, **kwargs)[source]

Get class for handling table-like formatted output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • fmt (str) – A C style format string specifying how each ‘row’ of output should be formated. This should include the newline character.

  • as_array (bool, optional) – If True, send expects and entire array. If False, send expects the entries for one table row. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggInit(_type, args=None, kwargs=None)[source]

Short interface to identify functions called by another language.

Parameters
  • _type (str) – Name of class that should be returned.

  • args (list, optional) – Additional arguments that should be passed to the class initializer.

  • kwargs (dict, optional) – Additional keyword arguments that should be passed to the class initializer.

Returns

An instance of the requested class.

Return type

obj

yggdrasil.languages.Python.YggInterface.YggInput(name, format_str=None, **kwargs)[source]

Get class for handling input from a message queue.

Parameters
  • name (str) – The name of the message queue. Combined with the suffix ‘_IN’, it should match an environment variable containing a message queue key.

  • format_str (str, optional) – C style format string that should be used to deserialize messages that are receieved into a list of python objects. Defaults to None and raw string messages are returned.

  • **kwargs – Additional keyword arguments are passed to the underlying comm.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggObjInput(name, **kwargs)[source]

Get class for handling Obj input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggObjOutput(name, **kwargs)[source]

Get class for handling Obj output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggOutput(name, format_str=None, **kwargs)[source]

Get class for handling output to a message queue.

Parameters
  • name (str) – The name of the message queue. Combined with the suffix ‘_OUT’, it should match an environment variable containing a message queue key.

  • format_str (str, optional) – C style format string that should be used to create a message from a list of python ojbects. Defaults to None and raw string messages are sent.

  • **kwargs – Additional keyword arguments are passed to the underlying comm.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPandasInput(name, **kwargs)[source]

Get class for handling Pandas input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPandasOutput(name, **kwargs)[source]

Get class for handling Pandas output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPickleInput(name, **kwargs)[source]

Get class for handling pickled input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPickleOutput(name, **kwargs)[source]

Get class for handling pickled output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPlyInput(name, **kwargs)[source]

Get class for handling Ply input.

Parameters
  • name (str) – The name of the input message queue that input should be received from.

  • **kwargs – Additional keyword arguments are passed to YggInput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggPlyOutput(name, **kwargs)[source]

Get class for handling Ply output.

Parameters
  • name (str) – The name of the message queue where output should be sent.

  • **kwargs – Additional keyword arguments are passed to YggOutput.

Returns

Communication object.

Return type

DefaultComm

yggdrasil.languages.Python.YggInterface.YggRpcClient(name, outfmt=None, infmt=None, **kwargs)[source]

Get class for handling requests and response to an RPC Server from a client.

Parameters
  • name (str) – The name of the server queues.

  • outfmt (str, optional) – Format string used to format variables in a message sent to the request queue. Defautls to ‘%s’.

  • infmt (str, optional) – Format string used to recover variables from messages received from the response queue. Defautls to ‘%s’.

  • **kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns

class:.ClientComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggRpcServer(name, infmt=None, outfmt=None, **kwargs)[source]

Get class for handling requests and response for an RPC Server.

Parameters
  • name (str) – The name of the server queues.

  • infmt (str, optional) – Format string used to recover variables from messages received from the request queue. Defaults to ‘%s’.

  • outfmt (str, optional) – Format string used to format variables in a message sent to the response queue. Defautls to ‘%s’.

  • **kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns

class:.ServerComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggTimesync(name='timesync', **kwargs)[source]

Get class for handling timestep synchronization requests.

Parameters
  • name (str, optional) – The name of the server queues. Defaults to ‘timesync’.

  • **kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns

class:.ClientComm: Communication object.

yggdrasil.languages.Python.YggInterface.YggTimesyncServer(name='timesync', **kwargs)[source]

Get class for handling timestep synchronization requests on the driver side.

Parameters
  • name (str, optional) – The name of the server queues. Defaults to ‘timesync’.

  • **kwargs – Additional keyword arguments are passed to InterfaceComm.

Returns

class:.ServerComm: Communication object.

yggdrasil.languages.Python.YggInterface.bufMsgSize()[source]

int: Buffer size for average message.

yggdrasil.languages.Python.YggInterface.eof_msg()[source]

str: Message signalling end of file.

yggdrasil.languages.Python.YggInterface.maxMsgSize()[source]

int: The maximum message size.

Module contents

yggdrasil.serialize package

Submodules

yggdrasil.serialize.AsciiMapSerialize module

class yggdrasil.serialize.AsciiMapSerialize.AsciiMapSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for serializing/deserializing name/value mapping.

Parameters

delimiter (str, optional) – Delimiter that should be used to separate name/value pairs in the map. Defaults to t.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = False
default_datatype = {'type': 'object'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (bytes) – Message to be deserialized.

Returns

Deserialized Python dictionary.

Return type

dict

func_serialize(args)[source]

Serialize a message.

Parameters

args (dict) – Python dictionary to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

yggdrasil.serialize.AsciiTableSerialize module

class yggdrasil.serialize.AsciiTableSerialize.AsciiTableSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.DefaultSerialize.DefaultSerialize

Class for serialize table output into bytes messages comprising a formatted ASCII table.

Parameters
  • format_str (str, optional) – If provided, this string will be used to format messages from a list of arguments and parse messages to get a list of arguments in C printf/scanf style. Defaults to None and messages are assumed to already be bytes.

  • field_names (list, optional) – The names of fields in the format string. If not provided, names are set based on the order of the fields in the format string.

  • field_units (list, optional) – The units of fields in the format string. If not provided, all fields are assumed to be dimensionless.

  • as_array (bool, optional) – If True, each of the arguments being serialized/deserialized will be arrays that are converted to/from bytes in column major (‘F’) order. Otherwise, each argument should be a scalar. Defaults to False.

  • delimiter (str, optional) – Character(s) that should be used to separate columns. Defaults to ‘t’.

  • use_astropy (bool, optional) – If True, the astropy package will be used to serialize/deserialize table. Defaults to False.

  • **kwargs – Additional keyword args are processed as part of the type definition.

format_str

Format string used to format/parse bytes messages from/to a list of arguments in C printf/scanf style.

Type

str

field_names

The names of fields in the format string.

Type

list

field_units

The units of fields in the format string.

Type

list

as_array

True or False depending if serialized/deserialized python objects will be arrays or scalars.

Type

bool

delimiter

Character(s) that should be used to separate columns.

Type

str

use_astropy

If True, the astropy package will be used to serialize/deserialize table.

Type

bool

default_datatype = {'type': 'array'}
default_read_meth = 'readline'
deserialize_file_header(fd)[source]

Deserialize the header information from the file and update the serializer.

Parameters

fd (file) – File containing header.

func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg – Message to be deserialized.

Returns

Deserialized message.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args – List of arguments to be formatted or numpy array to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Dictionary of variables to use for testing.

Return type

dict

has_header = True
property read_meth

Method that should be used to read data for deserialization.

Type

str

serialize_file_header()[source]

Return the serialized header information that should be prepended to files serialized using this class.

Returns

Header string that should be written to the file.

Return type

bytes

update_field_names()[source]

list: Names for each field in the data type.

update_field_units()[source]

list: Units for each field in the data type.

update_format_str()[source]

Update the format string based on the type definition.

update_serializer(*args, **kwargs)[source]

Update serializer with provided information.

Parameters
  • *args – All arguments are passed to the parent class’s method.

  • **kwargs – All keyword arguments are passed to the parent class’s method.

yggdrasil.serialize.DefaultSerialize module

class yggdrasil.serialize.DefaultSerialize.DefaultSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Default class for serializing/deserializing a python object into/from a bytes message.

Parameters

**kwargs – Additional keyword args are passed to the parent class.

classmethod concatenate(objects, as_array=False, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

func_deserialize = None
func_serialize = None
classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

classmethod object2array(obj, as_array=False, field_names=None, **kwargs)[source]

Convert a message object into an array.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in an array form.

  • as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.

  • field_names (list, optional) – The field names associated with a table-like data type. Defaults to None. This keyword must be provided if as_array is True.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Array version of the provided object.

Return type

np.array

classmethod object2dict(obj, as_array=False, field_names=None, **kwargs)[source]

Convert a message object into a dictionary.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.

  • as_array (bool, optional) – If True, the objects in the list are complete columns in a table and as_format is set to True. Defaults to False.

  • field_names (list, optional) – The field names associated with a table-like data type. Defaults to None. This keyword must be provided if as_array is True.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Dictionary version of the provided object.

Return type

dict

update_serializer(*args, **kwargs)[source]

Update serializer with provided information.

Parameters
  • *args – All arguments are passed to the parent class’s method.

  • **kwargs – All keyword arguments are passed to the parent class’s method.

yggdrasil.serialize.DirectSerialize module

class yggdrasil.serialize.DirectSerialize.DirectSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for directly serializing bytes.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

default_datatype = {'type': 'bytes'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg – Message to be deserialized.

Returns

Deserialized message.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args – List of arguments to be formatted or numpy array to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing. Key/value pairs:
kwargs (dict): Keyword arguments for comms tested with the

provided content.

empty (object): Object produced from deserializing an empty

message.

objects (list): List of objects to be serialized/deserialized. extra_kwargs (dict): Extra keyword arguments not used to

construct type definition.

typedef (dict): Type definition resulting from the supplied

kwargs.

dtype (np.dtype): Numpy data types that is consistent with the

determined type definition.

Return type

dict

yggdrasil.serialize.FunctionalSerialize module

class yggdrasil.serialize.FunctionalSerialize.FunctionalSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for serializing/deserializing a Python object into/from a bytes message using defined functions.

Parameters
  • encoded_datatype (schema, optional) – JSON schema describing the type that serialized objects should conform to. Defaults to the class attribute default_encoded_datatype. If either func_serialize or func_deserialize are not provided, this needs to be specified in order to serialize non-bytes objects.

  • func_serialize (func, optional) – Callable object that takes Python objects as input and returns a representation that conforms to encoded_datatype. Defaults to None and the default serialization for encoded_datatype will be used.

  • func_deserialize (func, optional) – Callable object that takes objects of a type that conforms to encoded_datatype and returns a deserialized Python object. Defaults to None and the default deserialization for encoded_datatype will be used.

  • **kwargs – Additional keyword args are passed to the parent class’s constructor.

property empty_msg

Object indicating empty message.

Type

obj

func_deserialize = None
func_serialize = None
property serializer_info

Serializer info.

Type

dict

yggdrasil.serialize.JSONSerialize module

class yggdrasil.serialize.JSONSerialize.JSONSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for serializing a python object into a bytes message using JSON.

Parameters
  • indent (str, int, optional) – String or number of spaces that should be used to indent each level within the seiralized structure. Defaults to ‘t’.

  • sort_keys (bool, optional) – If True, the serialization of dictionaries will be in key sorted order. Defaults to True.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = False
default_datatype = {'type': 'object'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (str, bytes) – Message to be deserialized.

Returns

Deserialized Python object.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args (obj) – Python object to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

yggdrasil.serialize.MatSerialize module

class yggdrasil.serialize.MatSerialize.MatSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for serializing a python object into a bytes message using the Matlab .mat format.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = False
default_datatype = {'type': 'object'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (str, bytes) – Message to be deserialized.

Returns

Deserialized Python object.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args (obj) – Python object to be serialized.

Returns

Serialized message.

Return type

bytes, str

Raises

TypeError – If args is not a dictionary.

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

yggdrasil.serialize.ObjSerialize module

class yggdrasil.serialize.ObjSerialize.ObjSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.PlySerialize.PlySerialize

Class for serializing/deserializing .obj file formats. Reader adapted from https://www.pygame.org/wiki/OBJFileLoader.

default_datatype = {'type': 'obj'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (bytes) – Message to be deserialized.

Returns

Deserialized message.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args – List of arguments to be formatted or numpy array to be serialized.

Returns

Serialized message.

Return type

bytes

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

yggdrasil.serialize.PandasSerialize module

class yggdrasil.serialize.PandasSerialize.PandasSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.AsciiTableSerialize.AsciiTableSerialize

Class for serializing/deserializing Pandas data frames.

Parameters
  • no_header (bool, optional) – If True, headers will not be read or serialized from/to tables. Defaults to False.

  • str_as_bytes (bool, optional) – If True, strings in columns are read as bytes. Defaults to False.

classmethod apply_field_names(frame, field_names=None)[source]

Apply field names as columns to a frame, first checking for a mapping. If there is a direct mapping, the columns are reordered to match the order of the field names. If there is not an overlap in the field names and columns, a one-to-one mapping is assumed, but a warning is issued. If there is a partial overlap, an error is raised.

Parameters
  • frame (pandas.DataFrame) – Frame to apply field names to as columns.

  • field_names (list, optional) – New field names that should be applied. If not provided, the original frame will be returned unaltered.

Returns

Frame with updated field names.

Return type

pandas.DataFrame

Raises

RuntimeError – If there is a partial overlap between the field names and columns.

as_array = True
cformat2nptype(*args, **kwargs)[source]

Method to convert c format string to numpy data type.

Parameters
  • *args – Arguments are passed to serialize.cformat2nptype.

  • **kwargs – Keyword arguments are passed to serialize.cformat2nptype.

Returns

Corresponding numpy data type.

Return type

np.dtype

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = False
consolidate_array(out)[source]

Consolidate message into a structure numpy array if possible.

Parameters

out (list, tuple, np.ndarray) – Object to consolidate into a structured numpy array.

Returns

Structured numpy array containing consolidated message.

Return type

np.ndarray

Raises

ValueError – If the array cannot be consolidated.

default_read_meth = 'read'
deserialize_file_header(fd)[source]

Deserialize the header information from the file and update the serializer.

Parameters

fd (file) – File containing header.

disable_file_header()[source]

Set serializer attributes to disable file headers from being included in the serializations.

property empty_msg

Object indicating empty message.

Type

obj

enable_file_header()[source]

Set serializer attributes to enable file headers to be included in the serializations.

func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (str, bytes) – Message to be deserialized.

Returns

Deserialized Python object.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args (obj) – Python object to be serialized.

Returns

Serialized message.

Return type

bytes, str

get_field_names(*args, **kwargs)[source]

Get the field names for an array of fields.

Parameters
  • *args – Arguments are passed to the parent class’s method.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Names for each field in the data type.

Return type

list

classmethod get_testing_options(not_as_frames=False, no_names=False, no_header=False, **kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters
  • not_as_frames (bool, optional) – If True, the returned example includes data that is not in a pandas data frame. Defaults to False.

  • no_names (bool, optional) – If True, an example is returned where the names are not provided to the deserializer. Defaults to False.

  • no_header (bool, optional) – If True, an example is returned

  • False. (where a header is not included. Defaults to) –

Returns

Dictionary of variables to use for testing.

Return type

dict

classmethod object2array(obj, **kwargs)[source]

Convert a message object into an array.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in an array form.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Array version of the provided object.

Return type

np.array

classmethod object2dict(obj, **kwargs)[source]

Convert a message object into a dictionary.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Dictionary version of the provided object.

Return type

dict

property send_converter
serialize_file_header()[source]

Return the serialized header information that should be prepended to files serialized using this class.

Returns

Header string that should be written to the file.

Return type

bytes

yggdrasil.serialize.PickleSerialize module

class yggdrasil.serialize.PickleSerialize.PickleSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.DefaultSerialize.DefaultSerialize

Class for serializing a python object into a bytes message by pickling.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (str, bytes) – Message to be deserialized.

Returns

Deserialized Python object.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args (obj) – Python object to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_first_frame(msg)[source]

Extract one frame from the provided message that may contain one or more frames.

Parameters

msg (bytes) – Message containing one or more frames.

Returns

Portion of message containing the first frame. If no frames

are found, an empty string will be returned.

Return type

bytes

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

is_framed = True

yggdrasil.serialize.PlySerialize module

class yggdrasil.serialize.PlySerialize.PlySerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.SerializeBase.SerializeBase

Class for serializing/deserializing .ply file formats.

Parameters
  • write_header (bool, optional) – If True, headers will be added to serialized output. Defaults to True.

  • newline (str, optional) – String that should be used for new lines. Defaults to ‘n’.

write_header

If True, headers will be added to serialized output.

Type

bool

newline

String that should be used for new lines.

Type

str

default_rgb

Default color in RGB that should be used for missing colors.

Type

list

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = False
default_datatype = {'type': 'ply'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (bytes) – Message to be deserialized.

Returns

Deserialized message.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args – List of arguments to be formatted or numpy array to be serialized.

Returns

Serialized message.

Return type

bytes

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

yggdrasil.serialize.SerializeBase module

class yggdrasil.serialize.SerializeBase.SerializeBase(*args, **kwargs)[source]

Bases: yggdrasil.tools.YggClass

Base class for serializing/deserializing a Python object into/from a bytes message.

Parameters
  • newline (str, optional) – One or more characters indicating a newline. Defaults to ‘n’.

  • comment (str, optional) – One or more characters indicating a comment. Defaults to ‘# ‘.

  • datatype (schema, optional) – JSON schema defining the type of object that the serializer will be used to serialize/deserialize. Defaults to default_datatype.

  • **kwargs – Additional keyword args are processed as part of the type definition.

initialized

True if the serializer has been initialized either by input arguments specifying the type or by infering the type from a processed message.

Type

bool

Class Attributes:
has_header (bool): True if the serialization has a header when written

to a file.

default_read_meth (str): Default method that data should be read from

a file for deserialization.

is_framed (bool): True if the serialization has a frame allowing

multiple serialized objects to be recovered from a single message.

concats_as_str (bool): True if serialized objects can be concatenated

directly as strings.

encoded_datatype (schema): JSON schema defining the type of object

produced by the class’s func_serialize method. For most classes this will be {‘type’: ‘bytes’}, indicating that the method will produce bytes suitable for serialization.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration.

cformat2nptype(*args, **kwargs)[source]

Method to convert c format string to numpy data type.

Parameters
  • *args – Arguments are passed to serialize.cformat2nptype.

  • **kwargs – Keyword arguments are passed to serialize.cformat2nptype.

Returns

Corresponding numpy data type.

Return type

np.dtype

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = True
consolidate_array(out)[source]

Consolidate message into a structure numpy array if possible.

Parameters

out (list, tuple, np.ndarray) – Object to consolidate into a structured numpy array.

Returns

Structured numpy array containing consolidated message.

Return type

np.ndarray

Raises

ValueError – If the array cannot be consolidated.

default_datatype = {'type': 'bytes'}
default_encoded_datatype = {'type': 'bytes'}
default_read_meth = 'read'
deserialize(msg, **kwargs)[source]

Deserialize a message.

Parameters
  • msg (str, bytes) – Message to be deserialized.

  • **kwargs – Additional keyword arguments are passed to the deserialize method of the datatype class.

Returns

Deserialized message and header information.

Return type

tuple(obj, dict)

Raises

TypeError – If msg is not bytes type (str on Python 2).

deserialize_file_header(fd)[source]

Deserialize the header information from the file and update the serializer.

Parameters

fd (file) – File containing header.

disable_file_header()[source]

Set serializer attributes to disable file headers from being included in the serializations.

property empty_msg

Object indicating empty message.

Type

obj

enable_file_header()[source]

Set serializer attributes to enable file headers to be included in the serializations.

property encoded_typedef

Type definition for encoded data objects.

Type

dict

func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg – Message to be deserialized.

Returns

Deserialized message.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args – List of arguments to be formatted or numpy array to be serialized.

Returns

Serialized message.

Return type

bytes, str

get_field_names(as_bytes=False)[source]

Get the field names for an array of fields.

Parameters

as_bytes (bool, optional) – If True, the field names will be returned as bytes. If False the field names will be returned as unicode. Defaults to False.

Returns

Names for each field in the data type.

Return type

list

get_field_units(as_bytes=False)[source]

Get the field units for an array of fields.

Parameters

as_bytes (bool, optional) – If True, the field units will be returned as bytes. If False the field units will be returned as unicode. Defaults to False.

Returns

Units for each field in the data type.

Return type

list

get_status_message(nindent=0, extra_lines_before=None, extra_lines_after=None)[source]

Return lines composing a status message.

Parameters
  • nindent (int, optional) – Number of tabs that should be used to indent each line. Defaults to 0.

  • extra_lines_before (list, optional) – Additional lines that should be added to the beginning of the default print message. Defaults to empty list if not provided.

  • extra_lines_after (list, optional) – Additional lines that should be added to the end of the default print message. Defaults to empty list if not provided.

Returns

Lines composing the status message and the

prefix string used for the last message.

Return type

tuple(list, prefix)

classmethod get_testing_options(table_example=False, array_columns=False, include_oldkws=False, table_string_type='bytes')[source]

Method to return a dictionary of testing options for this class.

Parameters
  • table_example (bool, optional) – If True, the returned options will be for an array of elements representing a table-like structure. Defaults to False.

  • array_columns (bool, optional) – If True, table_example is set to True and the returned options will be for an array data type where each element is an array representing a column Defaults to False.

  • include_oldkws (bool, optional) – If True, old-style keywords will be added to the returned options. This will only have an effect if table_example is True. Defaults to False.

  • table_string_type (str, optional) – Type that should be used for the string column in the table. Defaults to ‘bytes’.

Returns

Dictionary of variables to use for testing. Key/value pairs:

  • kwargs (dict): Keyword arguments for comms tested with the provided content.

  • empty (object): Object produced from deserializing an empty message.

  • objects (list): List of objects to be serialized/deserialized. extra_kwargs (dict): Extra keyword arguments not used to construct type definition.

  • typedef (dict): Type definition resulting from the supplied kwargs.

  • dtype (np.dtype): Numpy data types that is consistent with the determined type definition.

  • contents (bytes): Concatenated serialization that will result from deserializing the serialized objects.

  • contents_recv (list): List of objects that would be deserialized from contents.

Return type

dict

has_header = False
initialize_from_message(msg, **metadata)[source]

Initialize the serializer based on recieved message.

Parameters
  • msg (object) – Message that serializer should be initialized from.

  • **kwargs – Additional keyword arguments are treated as metadata that may contain additional information for initializing the serializer.

initialize_serializer(metadata, extract=False)[source]

Initialize a serializer based on received metadata. This method will exit early if the serializer has already been intialized.

Parameters
  • metadata (dict) – Header information including type info that should be used to initialize the serializer class.

  • extract (bool, optional) – If True, the type will be defined using a subset of the type information in metadata. If False, all of the type information will be used. Defaults to False.

property input_kwargs

Get the input keyword arguments used to create this class.

Type

dict

is_framed = False
is_initialized()[source]

Determine if the serializer has been initialized by comparing the current datatype against the default for the class.

Returns

True if the current datatype is different than the default,

False otherwise.

Return type

bool

property numpy_dtype

Corresponding structured data type. Will be None unless the type is an array of 1darrays.

Type

np.dtype

classmethod object2array(obj, **kwargs)[source]

Convert a message object into an array.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in an array form.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Array version of the provided object or None if one cannot

be created.

Return type

np.array

classmethod object2dict(obj, **kwargs)[source]

Convert a message object into a dictionary.

Parameters
  • obj (object) – Object that would be serialized by this class and should be returned in a dictionary form.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Dictionary version of the provided object.

Return type

dict

parse_header(msg)[source]

Extract header info from a message.

Parameters

msg (str) – Message to extract header from.

Returns

Message properties.

Return type

dict

property read_meth

Method that should be used to read data for deserialization.

Type

str

classmethod seri_kws()[source]

Get a list of valid keyword arguments.

serialize(args, header_kwargs=None, add_serializer_info=False, no_metadata=False, max_header_size=0)[source]

Serialize a message.

Parameters
  • args (obj) – List of arguments to be formatted or a ready made message.

  • header_kwargs (dict, optional) – Keyword arguments that should be added to the header. Defaults to None and no header is added.

  • add_serializer_info (bool, optional) – If True, serializer information will be added to the metadata. Defaults to False.

  • no_metadata (bool, optional) – If True, no metadata will be added to the serialized message. Defaults to False.

  • max_header_size (int, optional) – Maximum size that header should occupy in order to be sent in a single message. A value of 0 indicates that any size header is valid. Defaults to 0.

Returns

Serialized message.

Return type

bytes, str

Raises

TypeError – If returned msg is not bytes type (str on Python 2).

serialize_file_header()[source]

Return the serialized header information that should be prepended to files serialized using this class.

Returns

Header string that should be written to the file.

Return type

bytes

property serializer_info

Serializer info.

Type

dict

property typedef

Type definition.

Type

dict

update_serializer(extract=False, skip_type=False, **kwargs)[source]

Update serializer with provided information.

Parameters
  • extract (bool, optional) – If True, the updated typedef will be the bare minimum as extracted from total set of provided keywords, otherwise the entire set will be sued. Defaults to False.

  • skip_type (bool, optional) – If True, everything is updated except the data type. Defaults to False.

  • **kwargs – Additional keyword arguments are processed as part of they type definition and are parsed for old-style keywords.

Raises

RuntimeError – If there are keywords that are not valid typedef keywords (currect or old-style).

update_typedef_from_oldstyle(typedef)[source]

Update a given typedef using an old, table-style serialization spec. Existing typedef values are not overwritten and warnings are raised if the provided serialization spec is not compatible with the type definition.

Parameters

typedef (dict) – Type definition to update.

Returns

Updated typedef.

Return type

dict

yggdrasil.serialize.WOFOSTParamSerialize module

class yggdrasil.serialize.WOFOSTParamSerialize.WOFOSTParamSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.AsciiMapSerialize.AsciiMapSerialize

Class for serializing/deserializing WOFOST parameter files.

classmethod concatenate(objects, **kwargs)[source]

Concatenate objects to get object that would be recieved if the concatenated serialization were deserialized.

Parameters
  • objects (list) – Objects to be concatenated.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Set of objects that results from concatenating those provided.

Return type

list

concats_as_str = True
default_datatype = {'type': 'object'}
func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (bytes) – Message to be deserialized.

Returns

Deserialized Python dictionary.

Return type

dict

func_serialize(args)[source]

Serialize a message.

Parameters

args (dict) – Python dictionary to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options()[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

classmethod parse_units(x)[source]

Parse units.

Parameters

x (str) – Unit string.

Returns

Propertly formatted units.

Return type

str

yggdrasil.serialize.YAMLSerialize module

class yggdrasil.serialize.YAMLSerialize.YAMLSerialize(*args, **kwargs)[source]

Bases: yggdrasil.serialize.JSONSerialize.JSONSerialize

Class for serializing a python object into a bytes message using YAML.

Parameters
  • indent (str, int, optional) – String or number of spaces that should be used to indent each level within the seiralized structure. Defaults to ‘t’.

  • encoding (str, optional) – Encoding that should be used to serialize the object. Defaults to ‘utf-8’.

  • default_flow_style (bool, optional) – If True, nested collections will be serialized in the block style. If False, they will always be serialized in the flow style. See PyYAML Documentation.

func_deserialize(msg)[source]

Deserialize a message.

Parameters

msg (str, bytes) – Message to be deserialized.

Returns

Deserialized Python object.

Return type

obj

func_serialize(args)[source]

Serialize a message.

Parameters

args (obj) – Python object to be serialized.

Returns

Serialized message.

Return type

bytes, str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Returns

Dictionary of variables to use for testing.

Return type

dict

Module contents

yggdrasil.metaschema.datatypes package

Submodules

yggdrasil.metaschema.datatypes.AnyMetaschemaType module

class yggdrasil.metaschema.datatypes.AnyMetaschemaType.AnyMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type associated with a scalar.

cross_language_support = False
classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

description = 'A type allowing any value that is expresible in some type.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod encode_data_readable(obj, typedef)[source]

Encode an object’s data in a readable format that may not be decoded in exactly the same way.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod get_temptype(typedef, return_none=False)[source]

Extract temporary type from type definition.

Parameters
  • typedef (dict) – Type definition containing temporary type under the temptype key.

  • return_none (bool, optional) – If True, None will be returned if there is not a temptype in the provided typedef. Defaults to False.

Returns

Temporary type definition or None if not present.

Return type

dict

classmethod issubtype(t)[source]

Determine if this type is a subclass of the provided type.

Parameters

t (str) – Type name to check against.

Returns

True if this type is a subtype of the specified type t.

Return type

bool

metadata_properties = ['type', 'temptype']
name = 'any'
properties = ['type', 'title', 'temptype']
python_types = (<class 'object'>,)
classmethod transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict, optional) – Type definition that should be used to transform the object. Defaults to None.

Returns

Transformed object.

Return type

object

classmethod validate(*args, **kwargs)[source]

Validate an object to check if it could be of this type. For this type, the returned boolean will always be True.

yggdrasil.metaschema.datatypes.ArrayMetaschemaType module

class yggdrasil.metaschema.datatypes.ArrayMetaschemaType.NDArrayMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

Type associated with a scalar.

description = 'An ND array with or without units.'
metadata_properties = ['type', 'subtype', 'precision', 'units', 'shape']
name = 'ndarray'
properties = ['type', 'title', 'subtype', 'precision', 'units', 'shape']
python_types = (<class 'numpy.ndarray'>, <class 'unyt.array.unyt_array'>)
classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

class yggdrasil.metaschema.datatypes.ArrayMetaschemaType.OneDArrayMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

Type associated with a scalar.

description = 'A 1D array with or without units.'
metadata_properties = ['type', 'subtype', 'precision', 'units', 'length']
name = '1darray'
properties = ['type', 'title', 'subtype', 'precision', 'units', 'length']
python_types = (<class 'numpy.ndarray'>, <class 'unyt.array.unyt_array'>)
classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.ClassMetaschemaType module

class yggdrasil.metaschema.datatypes.ClassMetaschemaType.ClassMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type for evaluating classes.

cross_language_support = False
classmethod decode_data(obj, typedef, working_dir=None)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

  • working_dir (str, optional) – Working directory that should be used to make relative paths absolute. Defaults to None.

Returns

Decoded object.

Return type

object

description = 'Type for Python classes.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

encoded_type = 'string'
name = 'class'
classmethod normalize(obj, working_dir=None)[source]

Normalize an object, if possible, to conform to this type.

Parameters
  • obj (object) – Object to normalize.

  • working_dir (str, optional) – Working directory that should be used to make relative paths absolute. Defaults to None.

Returns

Normalized object.

Return type

object

python_types = (<class 'type'>,)
class yggdrasil.metaschema.datatypes.ClassMetaschemaType.ExampleClass(*args, **kwargs)[source]

Bases: object

yggdrasil.metaschema.datatypes.ContainerMetaschemaType module

class yggdrasil.metaschema.datatypes.ContainerMetaschemaType.ContainerMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type associated with a container of subtypes.

classmethod coerce_type(obj, typedef=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

Raises

RuntimeError – If obj is a dictionary, but key_order is not provided.

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

description = 'A container of other types.'
classmethod encode_data(obj, typedef, **kwargs)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

  • **kwargs – Additional keyword arguments are passed to the class method _encode_data_alias.

Returns

Encoded object.

Return type

string

classmethod encode_data_readable(obj, typedef, **kwargs)[source]

Encode an object’s data in a readable format.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

  • **kwargs – Additional keyword arguments are passed to the class method _encode_data_alias.

Returns

Encoded object.

Return type

string

classmethod extract_typedef(metadata)[source]

Extract the minimum typedef required for this type from the provided metadata.

Parameters

metadata (dict) – Message metadata.

Returns

Encoded type definition with unncessary properties removed.

Return type

dict

name = 'container'
python_types = []
classmethod transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict) – Type definition that should be used to transform the object.

Returns

Transformed object.

Return type

object

update_typedef(**kwargs)[source]

Update the current typedef with new values.

Parameters

**kwargs – All keyword arguments are considered to be new type definitions. If they are a valid definition property, they will be copied to the typedef associated with the instance.

Returns

A dictionary of keyword arguments that were not added to the

type definition.

Return type

dict

yggdrasil.metaschema.datatypes.FixedMetaschemaType module

class yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Class that should be used to alias another type, but with certain properties fixed.

Parameters

**kwargs – All keyword arguments are assumed to be type definition properties which will be used to validate serialized/deserialized messages.

fixed_properties

Type properties of the parent class that are fixed and the values they are fixed to.

Type

dict

classmethod base()[source]

Get the type that this type is a fixed version of.

classmethod check_encoded(metadata, typedef=None, raise_errors=False, **kwargs)[source]

Checks if the metadata for an encoded object matches the type definition.

Parameters
  • metadata (dict) – Meta data to be tested.

  • typedef (dict, optional) – Type properties that object should be tested against. Defaults to None and object may have any values for the type properties (so long as they match the schema.

  • raise_errors (bool, optional) – If True, any errors determining that encoded object is not of this type will be raised. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class.

Returns

True if the metadata matches the type definition, False

otherwise.

Return type

bool

classmethod extract_typedef(metadata)[source]

Extract the minimum typedef required for this type from the provided metadata.

Parameters

metadata (dict) – Message metadata.

Returns

Encoded type definition with unncessary properties removed.

Return type

dict

fixed_properties = {}
is_fixed = True
classmethod issubtype(t)[source]

Determine if this type is a subclass of the provided type.

Parameters

t (str) – Type name to check against.

Returns

True if this type is a subtype of the specified type t.

Return type

bool

classmethod typedef_base2fixed(typedef)[source]

Transform a type definition from the unfixed base type to the fixed type alias by removing fixed properties.

Parameters

typedef (dict) – Type definition for the unfixed base type that might include properties that are fixed in the base.

Returns

Copy of type definition with fixed properties removed.

Return type

dict

classmethod typedef_fixed2base(typedef)[source]

Transform a type definition from the fixed alias to the unfixed base by setting the fixed properties if they are not already present.

Parameters

typedef (dict) – Type definition for the fixed alias type.

Returns

Copy of type definition with fixed properties added.

Return type

dict

update_typedef(**kwargs)[source]

Update the current typedef with new values.

Parameters

**kwargs – All keyword arguments are considered to be new type definitions. If they are a valid definition property, they will be copied to the typedef associated with the instance.

Returns

A dictionary of keyword arguments that were not added to the

type definition.

Return type

dict

classmethod updated_fixed_properties(obj)[source]

Get a version of the fixed properties schema that includes information from the object.

Parameters

obj (object) – Object to use to put constraints on the fixed properties schema.

Returns

Fixed properties schema with object dependent constraints.

Return type

dict

classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.FixedMetaschemaType.create_fixed_type_class(name, description, base, fixed_properties, target_globals=None, class_name=None, **kwargs)[source]

Create a fixed class.

Parameters
  • name (str) –

  • description (str) – Description of the fixed type.

  • base (MetaschemaType) – Base class that should be used.

  • fixed_properties (dict) – Mapping between properties that are fixed and the value they are fixed to.

  • target_globals (dict, optional) – Globals dictionary for module where the fixed class should be added. If None, the new class is returned. Defaults to None.

  • class_name (str, optional) – Name that should be given to the class. If not provided, defaults to ‘<name.title()>MetaschemaType’.

  • **kwargs – Additional keyword arguments are treated as attributes that should be set on the fixed class.

Returns

The name of the class created if target_globals is provided,

the created class if target_globals is None.

Return type

str, class

yggdrasil.metaschema.datatypes.FunctionMetaschemaType module

class yggdrasil.metaschema.datatypes.FunctionMetaschemaType.FunctionMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.ClassMetaschemaType.ClassMetaschemaType

Type for evaluating functions.

description = 'Type for callable Python functions.'
name = 'function'
python_types = (<class 'builtin_function_or_method'>, <class 'function'>, <class 'builtin_function_or_method'>, <class 'method'>)
yggdrasil.metaschema.datatypes.FunctionMetaschemaType.example_func(x)[source]

yggdrasil.metaschema.datatypes.InstanceMetaschemaType module

class yggdrasil.metaschema.datatypes.InstanceMetaschemaType.InstanceMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type for evaluating instances of Python classes.

cross_language_support = False
classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type', 'class']
description = 'Type for Python class instances.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

extract_properties = ['type', 'title', 'class', 'args', 'kwargs']
classmethod get_test_data(typedef=None)[source]

object: Test data.

metadata_properties = ['type', 'class', 'args', 'kwargs']
name = 'instance'
properties = ['type', 'title', 'class', 'args', 'kwargs']
python_types = (<class 'object'>,)
classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.JSONArrayMetaschemaType module

class yggdrasil.metaschema.datatypes.JSONArrayMetaschemaType.JSONArrayMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.ContainerMetaschemaType.ContainerMetaschemaType

Type associated with a set of subtypes.

Developer Notes:

Support for dynamic arrays in C/C++ is still under development.

classmethod coerce_type(obj, typedef=None, key_order=None, dont_wrap_single=False, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • key_order (list, optional) – Order that keys from a dictionary should be used to compose an array. Defaults to None.

  • dont_wrap_single (bool, optional) – If True, single element data types will not attempt to wrap input object in additional list. Defaults to False.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

Raises

RuntimeError – If obj is a dictionary, but key_order is not provided.

description = 'A container of ordered values.'
classmethod encode_type(obj, **kwargs)[source]

Encode an object’s type definition.

Parameters
  • obj (object) – Object to encode.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Encoded type definition.

Return type

dict

extract_properties = ['type', 'title', 'items']
classmethod get_test_data(typedef=None)[source]

object: Test data.

metadata_properties = ['type', 'items']
name = 'array'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

properties = ['type', 'title', 'items']
python_types = (<class 'list'>, <class 'tuple'>, <class 'numpy.ndarray'>, <class 'pandas.core.frame.DataFrame'>)
classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.JSONMetaschemaType module

class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONBooleanMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase

JSON base boolean type.

description = 'JSON boolean type.'
example_data = True
name = 'boolean'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

python_types = (<class 'bool'>,)
class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONIntegerMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase

JSON base integer type.

cross_language_support = False
description = 'JSON integer type.'
example_data = 1
name = 'integer'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

python_types = (<class 'int'>,)
class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Base type for default JSON types.

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

description = 'A json base type.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

name = 'json'
specificity = -1
class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONNullMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase

JSON base null type.

description = 'JSON null type.'
example_data = None
name = 'null'
python_types = (<class 'NoneType'>,)
class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONNumberMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase

JSON base number type.

Developer Notes:

This covers the JSON default for floating point or integer values.

description = 'JSON number type.'
example_data = 1.0
name = 'number'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

python_types = (<class 'numbers.Number'>,)
class yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONStringMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.JSONMetaschemaType.JSONMetaschemaTypeBase

JSON base string type.

Developer Notes:

Encoding dependent on JSON library.

description = 'JSON string type.'
example_data = 'hello'
name = 'string'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

python_types = (<class 'str'>,)

yggdrasil.metaschema.datatypes.JSONObjectMetaschemaType module

class yggdrasil.metaschema.datatypes.JSONObjectMetaschemaType.JSONObjectMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.ContainerMetaschemaType.ContainerMetaschemaType

Type associated with a map.

Developer Notes:

Support for dynamic objects in C/C++ is still under development.

classmethod coerce_type(obj, typedef=None, key_order=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • key_order (list, optional) – Order or keys correpsonding to elements in a provided list or tuple. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

Raises

RuntimeError – If obj is a list or tuple, but key_order is not provided.

description = 'A container mapping between keys and values.'
extract_properties = ['type', 'title', 'properties']
classmethod get_test_data(typedef=None)[source]

object: Test data.

metadata_properties = ['type', 'properties']
name = 'object'
properties = ['type', 'title', 'properties']
python_types = (<class 'dict'>,)

yggdrasil.metaschema.datatypes.MetaschemaType module

class yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType(**typedef)[source]

Bases: object

Base type that should be subclassed by user defined types. Attributes should be overwritten to match the type.

Parameters

**kwargs – All keyword arguments are assumed to be type definition properties which will be used to validate serialized/deserialized messages.

Class Attributes:
name (str): Name of the type for use in YAML files & form options.

[REQUIRED]

description (str): A short description of the type. [REQUIRED] properties (list): List of JSON schema properties that this type uses.

These properties will be added to those form the paretn type class unless inherit_properties is False. Defaults to [‘type’, ‘title’].

definition_properties (list): Type properties that are required for YAML

or form entries specifying the type. These will also be used to validate type definitions. These properties will be added to those from the parent type class unless inherit_properties is False. Defaults to [‘type’].

metadata_properties (list): Type properties that are required for

deserializing instances of the type that have been serialized. These properties will be added to those from the parent class unless inherit_properties is False. Defaults to [‘type’].

extract_properties (list): Properties that will be extracted from the

metadata of a message to construct the type definition. These properties will be added to those from the parent class unless inherit_properties is False. Defaults to [‘type’, ‘title’].

python_types (list): List of python types that this type encompasses.

[REQUIRED].

specificity (int): Specificity of the type. Types with larger values are

more specific while types with smaller values are more general. Base types have a specificity of 0. More specific types are checked first before more general ones.

is_fixed (bool): True if the type is a fixed version of another type. See

FixedMetaschemaType for details.

inherit_properties (bool, list): If True, a type class’s properties will

be a combination of the parent class’s properties and those explicitly defined in the child class. If False, a type class’s properties will only be those explicitly defined in the child class. If a list of property attributes, only properties specified by those attributes will be inherited from the parent class.

schema_file (str, None): If set, the class’s schema is loaded from the

specified file at registration and moved to the loaded_schema_file class attribute. Defaults to None.

loaded_schema_file (str, None): The path to the file the schema for the

type was loaded from if it was loaded.

cross_language_support (bool): If True, this indicates the types should

be serializable across most (if not all) of the supported languages. If False, this type is not required to be serializable except to/from Python. Defaults to True.

classmethod check_decoded(obj, typedef=None, raise_errors=False, typedef_validated=False)[source]

Checks if an object is of the this type.

Parameters
  • obj (object) – Object to be tested.

  • typedef (dict, optional) – Type properties that object should be tested against. Defaults to None and is not used.

  • raise_errors (bool, optional) – If True, any errors determining that decoded object is not of this type will be raised. Defaults to False.

  • typedef_validated (bool, optional) – If True, the type definition is taken as already having been validated and will not be validated again during the encoding process. Defaults to False.

Returns

Truth of if the input object is of this type.

Return type

bool

classmethod check_encoded(metadata, typedef=None, raise_errors=False, typedef_validated=False, metadata_validated=False)[source]

Checks if the metadata for an encoded object matches the type definition.

Parameters
  • metadata (dict) – Meta data to be tested.

  • typedef (dict, optional) – Type properties that object should be tested against. Defaults to None and object may have any values for the type properties (so long as they match the schema.

  • raise_errors (bool, optional) – If True, any errors determining that encoded object is not of this type will be raised. Defaults to False.

  • typedef_validated (bool, optional) – If True, the type definition is taken as already having been validated and will not be validated again during the encoding process. Defaults to False.

  • metadata_validated (bool, optional) – If True, the metadata definition is taken as already having been valdiated and will not be validated again. Defaults to False.

Returns

True if the metadata matches the type definition, False

otherwise.

Return type

bool

classmethod coerce_type(obj, typedef=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

cross_language_support = True
classmethod decode(metadata, data, typedef=None, typedef_validated=False, dont_check=False)[source]

Decode an object.

Parameters
  • metadata (dict) – Meta data describing the data.

  • data (bytes) – Encoded data.

  • typedef (dict, optional) – Type properties that decoded object should be tested against. Defaults to None and object may have any values for the type properties (so long as they match the schema).

  • typedef_validated (bool, optional) – If True, the type definition is taken as already having been validated and will not be validated again during the encoding process. Defaults to False.

  • dont_check (bool, optional) – If True, the metadata will not be checked against the type definition. Defaults to False.

Returns

Decoded object.

Return type

object

Raises
  • ValueError – If the metadata does not match the type definition.

  • ValueError – If the decoded object does not match type definition.

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type']
classmethod definition_schema()[source]

JSON schema for validating a type definition schema.

description = 'A generic base type for users to build on.'
deserialize(msg, no_data=False, metadata=None, dont_decode=False, dont_check=False)[source]

Deserialize a message.

Parameters
  • msg (str, bytes) – Message to be deserialized.

  • no_data (bool, optional) – If True, only the metadata is returned. Defaults to False.

  • metadata (dict, optional) – Metadata that should be used to deserialize the message instead of the current header content. Defaults to None and is not used.

  • dont_decode (bool, optional) – If True, type specific and JSON decoding will not be used to decode the message. Defaults to False.

  • dont_check (bool, optional) – If True, the metadata will not be checked against the type definition. Defaults to False.

Returns

Deserialized message and header information.

Return type

tuple(obj, dict)

Raises
  • TypeError – If msg is not bytes type (str on Python 2).

  • ValueError – If msg does not contain the header separator.

classmethod encode(obj, typedef=None, typedef_validated=False, dont_check=False, **kwargs)[source]

Encode an object.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict, optional) – Type properties that object should be tested against. Defaults to None and object may have any values for the type properties (so long as they match the schema.

  • typedef_validated (bool, optional) – If True, the type definition is taken as already having been validated and will not be validated again during the encoding process. Defaults to False.

  • dont_check (bool, optional) – If True, the object will not be checked against the type definition. Defaults to False.

  • **kwargs – Additional keyword arguments are added to the metadata.

Returns

Encoded object with type definition and data

serialized to bytes.

Return type

tuple(dict, bytes)

Raises
  • ValueError – If the object does not match the type definition.

  • ValueError – If the encoded metadata does not match the type definition.

  • TypeError – If the encoded data is not of bytes type.

classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod encode_data_readable(obj, typedef)[source]

Encode an object’s data in a readable format.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod encode_type(obj, typedef=None, is_validated=False, **kwargs)[source]

Encode an object’s type definition.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict, optional) – Type properties that should be used to initialize the encoded type definition in certain cases. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are treated as additional schema properties.

Raises

MetaschemaTypeError – If the object is not the correct type.

Returns

Encoded type definition.

Return type

dict

extract_properties = ['type', 'title']
classmethod extract_typedef(metadata, reqkeys=None)[source]

Extract the minimum typedef required for this type from the provided metadata.

Parameters
  • metadata (dict) – Message metadata.

  • reqkeys (list, optional) – Set of keys to keep in the definition. Defaults to the required definition keys.

Returns

Encoded type definition with unncessary properties removed.

Return type

dict

classmethod generate_data(typedef, **kwargs)[source]

Generate mock data for the specified type.

Parameters

typedef (dict) – Type definition.

Returns

Python object of the specified type.

Return type

object

classmethod get_extract_properties(metadata)[source]

Get the list of properties that should be kept when extracting a typedef from message metadata.

Parameters

metadata (dict) – Metadata that typedef is being extracted from.

Returns

Keywords that should be kept in the typedef.

Return type

list

classmethod get_test_data(typedef=None)[source]

object: Test data.

inherit_properties = True
is_fixed = False
classmethod issubtype(t)[source]

Determine if this type is a subclass of the provided type.

Parameters

t (str, list) – Type name or list of type names to check against.

Returns

True if this type is a subtype of the specified type t.

Return type

bool

classmethod jsonschema_type_checker(checker, instance)[source]

Type checker for use with jsonschema >= 3.0.0.

Parameters
  • checker (jsonschema.TypeChecker) – Type checker class.

  • instance (object) – Object being checked.

Returns

True if instance could be of this type, False otherwise.

Return type

bool

loaded_schema_file = None
metadata_properties = ['type']
classmethod metadata_schema()[source]

JSON schema for validating a JSON serialization of the type.

classmethod metaschema()[source]

JSON meta schema for validating schemas for this type.

name = 'base'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

classmethod normalize_definition(obj)[source]

Normalizes a type definition.

Parameters

obj (object) – Type definition to normalize.

Returns

Normalized type definition.

Return type

object

properties = ['type', 'title']
python_types = []
schema_file = None
serialize(obj, no_metadata=False, dont_encode=False, dont_check=False, max_header_size=0, **kwargs)[source]

Serialize a message.

Parameters
  • obj (object) – Python object to be formatted.

  • no_metadata (bool, optional) – If True, no metadata will be added to the serialized message. Defaults to False.

  • dont_encode (bool, optional) – If True, the input message will not be encoded using type specific or JSON encoding. Defaults to False.

  • dont_check (bool, optional) – If True, the object being serialized will not be checked against the type definition. Defaults to False.

  • max_header_size (int, optional) – Maximum size that header should occupy in order to be sent in a single message. A value of 0 indicates that any size header is valid. Defaults to 0.

  • **kwargs – Additional keyword arguments are added to the metadata.

Returns

Serialized message.

Return type

bytes, str

specificity = 0
classmethod transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict) – Type definition that should be used to transform the object.

Returns

Transformed object.

Return type

object

update_typedef(**kwargs)[source]

Update the current typedef with new values.

Parameters

**kwargs – All keyword arguments are considered to be new type definitions. If they are a valid definition property, they will be copied to the typedef associated with the instance.

Returns

A dictionary of keyword arguments that were not added to the

type definition.

Return type

dict

Raises

MetaschemaTypeError – If the current type does not match the type being updated to.

classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

classmethod validate_definition(obj, **kwargs)[source]

Validates a type definition.

Parameters
  • obj (object) – Type definition to validate.

  • **kwargs – Additional keyword arguments are passed to the validator.

classmethod validate_instance(obj, typedef, **kwargs)[source]

Validates an object against a type definition.

Parameters
  • obj (object) – Object to validate against a type definition.

  • typedef (dict) – Type definition to validate against.

  • **kwargs – Additional keyword arguments are passed to the validator.

classmethod validate_metadata(obj, **kwargs)[source]

Validates an encoded object.

Parameters
  • obj (string) – Encoded object to validate.

  • **kwargs – Additional keyword arguments are passed to the validator.

classmethod validator()[source]

JSON schema validator for the meta schema that includes added types.

yggdrasil.metaschema.datatypes.MultiMetaschemaType module

class yggdrasil.metaschema.datatypes.MultiMetaschemaType.MultiMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type class for handling behavior when more than one type is valid.

classmethod coerce_type(obj, typedef=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

classmethod definition_schema()[source]

JSON schema for validating a type definition schema.

classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod encode_type(obj, typedef=None, **kwargs)[source]

Encode an object’s type definition.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict, optional) – Type properties that should be used to initialize the encoded type definition in certain cases. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are treated as additional schema properties.

Returns

Encoded type definition.

Return type

dict

classmethod get_type_class(typedef=None, obj=None)[source]

Get the type class from the provided typedef.

inherit_properties = False
classmethod issubtype(t)[source]

Determine if this type is a subclass of the provided type.

Parameters

t (str, list) – Type name or list of type names to check against.

Returns

True if this type is a subtype of the specified type t.

Return type

bool

classmethod metadata_schema()[source]

JSON schema for validating a JSON serialization of the type.

classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

classmethod transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict) – Type definition that should be used to transform the object.

Returns

Transformed object.

Return type

object

type_classes = {}
update_typedef(**kwargs)[source]

Update the current typedef with new values.

Parameters

**kwargs – All keyword arguments are considered to be new type definitions. If they are a valid definition property, they will be copied to the typedef associated with the instance.

Returns

A dictionary of keyword arguments that were not added to the

type definition.

Return type

dict

Raises

MetaschemaTypeError – If the current type does not match the type being updated to.

classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.MultiMetaschemaType.create_multitype_class(types)[source]

Create a MultiMetaschemaType class that wraps multiple classes.

Parameters

types (list) – List of names of types.

Returns

Subclass of MultiMetaschemaType that add classes.

Return type

class

yggdrasil.metaschema.datatypes.ObjMetaschemaType module

class yggdrasil.metaschema.datatypes.ObjMetaschemaType.ObjDict(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.PlyMetaschemaType.PlyDict

Enhanced dictionary class for storing Obj information.

append(solf)[source]

Append new ply information to this dictionary.

Parameters

solf (ObjDict) – Another ply to append to this one.

apply_scalar_map(*args, **kwargs)[source]

Set the color of faces in a 3D object based on a scalar map. This creates a copy unless no_copy is True.

Parameters
  • scalar_arr (arr) – Scalar values that should be mapped to colors for each face.

  • color_map (str, optional) – The name of the color map that should be used. Defaults to ‘plasma’.

  • vmin (float, optional) – Value that should map to the minimum of the colormap. Defaults to min(scalar_arr).

  • vmax (float, optional) – Value that should map to the maximum of the colormap. Defaults to max(scalar_arr).

  • scaling (str, optional) – Scaling that should be used to map the scalar array onto the colormap. Defaults to ‘linear’.

  • scale_by_area (bool, optional) – If True, the elements of the scalar array will be multiplied by the area of the corresponding face. If True, vmin and vmax should be in terms of the scaled array. Defaults to False.

  • no_copy (bool, optional) – If True, the returned object will not be a copy. Defaults to False.

Returns

Obj with updated vertex colors.

Return type

dict

as_array_dict()[source]

Get a version of the object as a dictionary of arrays.

as_trimesh(**kwargs)[source]

Get a version of the object as a trimesh class.

classmethod from_array_dict(in_dict)[source]

Get a version of the object from a dictionary of arrays.

classmethod from_shape(shape, d, conversion=1.0)[source]

Create a ply dictionary from a PlantGL shape and descritizer.

Parameters
  • scene (openalea.plantgl.scene) – Scene that should be descritized.

  • d (openalea.plantgl.descritizer) – Descritizer.

  • conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

classmethod from_trimesh(in_mesh)[source]

Get a version of the object from a trimesh class.

property mesh

Vertices for each face in the structure.

Type

list

to_geom_args(conversion=1.0, name=None)[source]

Get arguments for creating a PlantGL geometry.

Parameters
  • conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.

  • name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns

Class, arguments and keyword arguments for PlantGL geometry.

Return type

tuple

property vertex_normals
class yggdrasil.metaschema.datatypes.ObjMetaschemaType.ObjMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.JSONObjectMetaschemaType.JSONObjectMetaschemaType

Obj 3D structure map.

classmethod coerce_type(obj, typedef=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

classmethod decode_data(msg, typedef)[source]

Decode an object.

Parameters
  • msg (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type']
description = 'A mapping container for Obj 3D data.'
classmethod encode_data(obj, typedef, comments=[], newline='\n')[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

  • comments (list, optional) – List of comments that should be included in the file header. Defaults to lines describing the automated origin of the file.

  • newline (str, optional) – String that should be used to delineated end of lines. Defaults to ‘n’.

Returns

Serialized message.

Return type

bytes, str

classmethod encode_data_readable(obj, typedef)[source]

Encode an object’s data in a readable format.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

fixed_properties = {'definitions': {'curve': {'additionalProperties': False, 'description': 'Properties of describing a curve.', 'properties': {'ending_param': {'type': 'float'}, 'starting_param': {'type': 'float'}, 'vertex_indices': {'items': {'type': ['int', 'uint']}, 'minItems': 2, 'type': 'array'}}, 'required': ['starting_param', 'ending_param', 'vertex_indices'], 'type': 'object'}, 'curve2D': {'description': 'Array of parameter indices describine a 2D curve on a surface.', 'items': {'type': ['int', 'uint']}, 'minItems': 2, 'type': 'array'}, 'face': {'description': 'Array of vertex, texture, and normal indices describing a face.', 'items': {'additionalProperties': False, 'properties': {'normal_index': {'type': ['int', 'uint']}, 'texcoord_index': {'type': ['int', 'uint']}, 'vertex_index': {'type': ['int', 'uint']}}, 'required': ['vertex_index'], 'type': 'object'}, 'minItems': 3, 'type': 'array'}, 'line': {'description': 'Array of vertex indices and texture indices describing a line.', 'items': {'additionalProperties': False, 'properties': {'texcoord_index': {'type': ['int', 'uint']}, 'vertex_index': {'type': ['int', 'uint']}}, 'required': ['vertex_index'], 'type': 'object'}, 'minItems': 2, 'type': 'array'}, 'normal': {'additionalProperties': False, 'description': 'Map describing a single normal.', 'properties': {'i': {'type': 'float'}, 'j': {'type': 'float'}, 'k': {'type': 'float'}}, 'required': ['i', 'j', 'k'], 'type': 'object'}, 'param': {'additionalProperties': False, 'description': 'Map describing a single parameter space point.', 'properties': {'u': {'type': 'float'}, 'v': {'type': 'float'}, 'w': {'default': 1.0, 'type': 'float'}}, 'required': ['u', 'v'], 'type': 'object'}, 'point': {'description': 'Array of vertex indices describing a set of points.', 'items': {'type': ['int', 'uint']}, 'minItems': 1, 'type': 'array'}, 'surface': {'additionalProperties': False, 'description': 'Properties describing a surface.', 'properties': {'ending_param_u': {'type': 'float'}, 'ending_param_v': {'type': 'float'}, 'starting_param_u': {'type': 'float'}, 'starting_param_v': {'type': 'float'}, 'vertex_indices': {'items': {'additionalProperties': False, 'properties': {'normal_index': {'type': ['int', 'uint']}, 'texcoord_index': {'type': ['int', 'uint']}, 'vertex_index': {'type': ['int', 'uint']}}, 'required': ['vertex_index'], 'type': 'object'}, 'minItems': 2, 'type': 'array'}}, 'required': ['starting_param_u', 'ending_param_u', 'starting_param_v', 'ending_param_v', 'vertex_indices'], 'type': 'object'}, 'texcoord': {'additionalProperties': False, 'description': 'Map describing a single texture vertex.', 'properties': {'u': {'type': 'float'}, 'v': {'default': 0.0, 'type': 'float'}, 'w': {'default': 0.0, 'type': 'float'}}, 'required': ['u'], 'type': 'object'}, 'vertex': {'additionalProperties': False, 'description': 'Map describing a single vertex.', 'properties': {'blue': {'type': ['int', 'uint']}, 'green': {'type': ['int', 'uint']}, 'red': {'type': ['int', 'uint']}, 'w': {'default': 1.0, 'type': 'float'}, 'x': {'type': 'float'}, 'y': {'type': 'float'}, 'z': {'type': 'float'}}, 'required': ['x', 'y', 'z'], 'type': 'object'}}, 'dependencies': {'curve2Ds': ['params'], 'curves': ['vertices'], 'faces': ['vertices'], 'lines': ['vertices'], 'surfaces': ['vertices']}, 'description': 'A mapping container for Obj 3D data.', 'properties': {'curve2Ds': {'description': 'Array of curve2Ds.', 'items': {'$ref': '#/definitions/curve2D'}, 'type': 'array'}, 'curves': {'description': 'Array of curves.', 'items': {'$ref': '#/definitions/curve'}, 'type': 'array'}, 'faces': {'description': 'Array of faces.', 'items': {'$ref': '#/definitions/face'}, 'type': 'array'}, 'lines': {'description': 'Array of lines.', 'items': {'$ref': '#/definitions/line'}, 'type': 'array'}, 'material': {'description': 'Name of the material to use.', 'type': ['unicode', 'string']}, 'normals': {'description': 'Array of normals.', 'items': {'$ref': '#/definitions/normal'}, 'type': 'array'}, 'params': {'description': 'Array of parameter coordinates.', 'items': {'$ref': '#/definitions/param'}, 'type': 'array'}, 'points': {'description': 'Array of points.', 'items': {'$ref': '#/definitions/point'}, 'type': 'array'}, 'surfaces': {'description': 'Array of surfaces.', 'items': {'$ref': '#/definitions/surface'}, 'type': 'array'}, 'texcoords': {'description': 'Array of texture vertices.', 'items': {'$ref': '#/definitions/texcoord'}, 'type': 'array'}, 'vertices': {'description': 'Array of vertices.', 'items': {'$ref': '#/definitions/vertex'}, 'type': 'array'}}, 'required': ['vertices', 'faces'], 'title': 'obj', 'type': 'object'}
loaded_schema_file = '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/metaschema/datatypes/schemas/obj.json'
metadata_properties = ['type']
name = 'obj'
properties = ['type']
python_types = (<class 'dict'>, <class 'yggdrasil.metaschema.datatypes.ObjMetaschemaType.ObjDict'>, <class 'trimesh.base.Trimesh'>)
specificity = 1
classmethod updated_fixed_properties(obj)[source]

Get a version of the fixed properties schema that includes information from the object.

Parameters

obj (object) – Object to use to put constraints on the fixed properties schema.

Returns

Fixed properties schema with object dependent constraints.

Return type

dict

yggdrasil.metaschema.datatypes.ObjMetaschemaType.create_schema(overwrite=False)[source]

Creates a file containing the Obj schema.

Parameters

overwrite (bool, optional) – If True and a file already exists, the existing file will be replaced. If False, an error will be raised if the file already exists.

yggdrasil.metaschema.datatypes.ObjMetaschemaType.get_schema()[source]

Return the Obj schema, initializing it if necessary.

Returns

Obj schema.

Return type

dict

yggdrasil.metaschema.datatypes.PlyMetaschemaType module

class yggdrasil.metaschema.datatypes.PlyMetaschemaType.PlyDict(*args, **kwargs)[source]

Bases: dict

Enhanced dictionary class for storing Ply information.

append(solf)[source]

Append new ply information to this dictionary.

Parameters

solf (PlyDict) – Another ply to append to this one.

apply_scalar_map(scalar_arr, color_map=None, vmin=None, vmax=None, scaling='linear', scale_by_area=False, no_copy=False, _as_obj=False)[source]

Set the color of faces in a 3D object based on a scalar map. This creates a copy unless no_copy is True.

Parameters
  • scalar_arr (arr) – Scalar values that should be mapped to colors for each face.

  • color_map (str, optional) – The name of the color map that should be used. Defaults to ‘plasma’.

  • vmin (float, optional) – Value that should map to the minimum of the colormap. Defaults to min(scalar_arr).

  • vmax (float, optional) – Value that should map to the maximum of the colormap. Defaults to max(scalar_arr).

  • scaling (str, optional) – Scaling that should be used to map the scalar array onto the colormap. Defaults to ‘linear’.

  • scale_by_area (bool, optional) – If True, the elements of the scalar array will be multiplied by the area of the corresponding face. If True, vmin and vmax should be in terms of the scaled array. Defaults to False.

  • no_copy (bool, optional) – If True, the returned object will not be a copy. Defaults to False.

Returns

Ply with updated vertex colors.

Return type

dict

as_array_dict()[source]

Get a version of the object as a dictionary of arrays.

as_dict()[source]

Get a version of the object as a pure dictionary.

as_trimesh(**kwargs)[source]

Get a version of the object as a trimesh class.

property bounds

Mins/maxs of vertices in each dimension.

Type

tuple

convert_arrays()[source]

Check fields and convert arrays to nested structures.

count_elements(element_name)[source]

Get the count of a certain element in the dictionary.

Parameters

element_name (str) – Name of the element to count.

Returns

Number of the provided element.

Return type

int

classmethod from_array_dict(in_dict)[source]

Get a version of the object from a dictionary of arrays.

classmethod from_dict(in_dict)[source]

Get a version of the object from a dictionary.

classmethod from_scene(scene, d=None, conversion=1.0)[source]

Create a ply dictionary from a PlantGL scene and descritizer.

Parameters
  • scene (openalea.plantgl.scene) – Scene that should be descritized.

  • d (openalea.plantgl.descritizer, optional) – Descritizer. Defaults to openalea.plantgl.all.Tesselator.

  • conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

classmethod from_shape(shape, d, conversion=1.0, _as_obj=False)[source]

Create a ply dictionary from a PlantGL shape and descritizer.

Parameters
  • scene (openalea.plantgl.scene) – Scene that should be descritized.

  • d (openalea.plantgl.descritizer) – Descritizer.

  • conversion (float, optional) – Conversion factor that should be applied to the vertex positions. Defaults to 1.0.

classmethod from_trimesh(in_mesh)[source]

Get a version of the object from a trimesh class.

merge(ply_list, no_copy=False)[source]

Merge a list of ply dictionaries.

Parameters
  • ply_list (list) – Ply dictionaries.

  • no_copy (bool, optional) – If True, the current dictionary will be updated, otherwise a copy will be returned with the update. Defaults to False.

Returns

Merged ply dictionary.

Return type

dict

property mesh

Vertices for each face in the structure.

Type

list

property nface

Number of faces.

Type

int

property nvert

Number of vertices.

Type

int

to_geom_args(conversion=1.0, name=None, _as_obj=False)[source]

Get arguments for creating a PlantGL geometry.

Parameters
  • conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.

  • name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns

Class, arguments and keyword arguments for PlantGL geometry.

Return type

tuple

to_scene(conversion=1.0, name=None)[source]

Create a PlantGL scene from a Ply dictionary.

Parameters
  • conversion (float, optional) – Conversion factor that should be applied to the vertices. Defaults to 1.0.

  • name (str, optional) – Name that should be given to the created PlantGL symbol. Defaults to None and is ignored.

Returns:

class yggdrasil.metaschema.datatypes.PlyMetaschemaType.PlyMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.JSONObjectMetaschemaType.JSONObjectMetaschemaType

Ply 3D structure map.

classmethod coerce_type(obj, typedef=None, **kwargs)[source]

Coerce objects of specific types to match the data type.

Parameters
  • obj (object) – Object to be coerced.

  • typedef (dict, optional) – Type defintion that object should be coerced to. Defaults to None.

  • **kwargs – Additional keyword arguments are metadata entries that may aid in coercing the type.

Returns

Coerced object.

Return type

object

classmethod decode_data(msg, typedef)[source]

Decode an object.

Parameters
  • msg (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type']
description = 'A mapping container for Ply 3D data.'
classmethod encode_data(obj, typedef, element_order=None, property_order=None, default_rgb=[0, 0, 0], comments=[], newline='\n', plyformat='ascii 1.0')[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

  • element_order (list, optional) – Order that elements should be written to the file. If not provided, the order is determined based on typical ply files with remaining elements output in sorted order.

  • property_order (dict, optional) – Dictionary of property order for each element determining the order that they properties should be written to the file. If not provided, the orders are determined based on typical ply files with remaining elements output in sorted order.

  • default_rgb (list, optional) – Default color in RGB that should be used for missing colors. Defaults to [0, 0, 0].

  • comments (list, optional) – List of comments that should be included in the file header. Defaults to lines describing the automated origin of the file.

  • newline (str, optional) – String that should be used to delineated end of lines. Defaults to ‘n’.

  • plyformat (str, optional) – String describing the ply format and version. Defaults to ‘ascii 1.0’.

Returns

Serialized message.

Return type

bytes, str

classmethod encode_data_readable(obj, typedef)[source]

Encode an object’s data in a readable format.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

fixed_properties = {'definitions': {'color': {'type': ['int', 'uint']}, 'coord': {'type': 'float'}, 'edge': {'additionalProperties': False, 'description': 'Vertex indices describing an edge.', 'properties': {'blue': {'type': ['int', 'uint']}, 'green': {'type': ['int', 'uint']}, 'red': {'type': ['int', 'uint']}, 'vertex1': {'type': ['int', 'uint']}, 'vertex2': {'type': ['int', 'uint']}}, 'required': ['vertex1', 'vertex2'], 'type': 'object'}, 'face': {'additionalProperties': False, 'description': 'Map describing a single face.', 'properties': {'vertex_index': {'items': {'type': ['int', 'uint']}, 'minItems': 3, 'type': 'array'}}, 'required': ['vertex_index'], 'type': 'object'}, 'index': {'type': ['int', 'uint']}, 'vertex': {'additionalProperties': False, 'description': 'Map describing a single vertex.', 'properties': {'blue': {'type': ['int', 'uint']}, 'green': {'type': ['int', 'uint']}, 'red': {'type': ['int', 'uint']}, 'x': {'type': 'float'}, 'y': {'type': 'float'}, 'z': {'type': 'float'}}, 'required': ['x', 'y', 'z'], 'type': 'object'}}, 'dependencies': {'edges': ['vertices'], 'faces': ['vertices']}, 'description': 'A mapping container for Ply 3D data.', 'properties': {'edges': {'description': 'Array of edges.', 'items': {'$ref': '#/definitions/edge'}, 'type': 'array'}, 'faces': {'description': 'Array of faces.', 'items': {'$ref': '#/definitions/face'}, 'type': 'array'}, 'material': {'description': 'Name of the material to use.', 'type': ['unicode', 'string']}, 'vertices': {'description': 'Array of vertices.', 'items': {'$ref': '#/definitions/vertex'}, 'type': 'array'}}, 'required': ['vertices', 'faces'], 'title': 'ply', 'type': 'object'}
loaded_schema_file = '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/metaschema/datatypes/schemas/ply.json'
metadata_properties = ['type']
name = 'ply'
properties = ['type']
python_types = (<class 'dict'>, <class 'yggdrasil.metaschema.datatypes.PlyMetaschemaType.PlyDict'>, <class 'trimesh.base.Trimesh'>)
specificity = 1
classmethod updated_fixed_properties(obj)[source]

Get a version of the fixed properties schema that includes information from the object.

Parameters

obj (object) – Object to use to put constraints on the fixed properties schema.

Returns

Fixed properties schema with object dependent constraints.

Return type

dict

yggdrasil.metaschema.datatypes.PlyMetaschemaType.create_schema(overwrite=False)[source]

Creates a file containing the Ply schema.

Parameters

overwrite (bool, optional) – If True and a file already exists, the existing file will be replaced. If False, an error will be raised if the file already exists.

yggdrasil.metaschema.datatypes.PlyMetaschemaType.get_key_order(all_keys, default_order)[source]

Determine the order of keys based on the keys and default order. Keys are added first in the default order and then alphabetically.

Parameters
  • all_keys (list) – List of all keys that should be present in the returned list.

  • default_order (list) – Default order for keys that may or may not be in all_keys.

Returns

Key order.

Return type

list

yggdrasil.metaschema.datatypes.PlyMetaschemaType.get_schema()[source]

Return the Ply schema, initializing it if necessary.

Returns

Ply schema.

Return type

dict

yggdrasil.metaschema.datatypes.PlyMetaschemaType.plural2singular(e_plur)[source]

Get the singular version of a plural element name. If the plural version ends with the suffix ‘ices’, it is replaced with the singular suffix ‘ex’. If the plural version ends with an ‘s’, it is removed.

Parameters

e_plur (str) – Plural version of an element name.

Returns

Singular version of the plural element name e_plur.

Return type

str

Raises

ValueError – If a singular version cannot be determined.

yggdrasil.metaschema.datatypes.PlyMetaschemaType.singular2plural(e_sing)[source]

Get the plural version of a singular element name. If the singular version ends with the suffix ‘ex’ it is replaced with the plural suffix ‘ices’. Otherwise, an ‘s’ is appended to the singular name to make it plural.

Parameters

e_sing (str) – Singular version of an element name.

Returns

Plural version of the singular element name e_sing.

Return type

str

yggdrasil.metaschema.datatypes.PlyMetaschemaType.translate_ply2fmt(type_ply)[source]

Get the corresponding type string for a Ply type string.

Parameters

type_ply (str) – Ply type string.

Returns

C-style format string.

Return type

str

yggdrasil.metaschema.datatypes.PlyMetaschemaType.translate_ply2py(type_ply)[source]

Get the corresponding Python type for the Ply type string.

Parameters

type_ply (str) – Ply type string.

Returns

Python type.

Return type

type

Raises

ValueError – If the type string does not have a match.

yggdrasil.metaschema.datatypes.PlyMetaschemaType.translate_py2ply(py_obj)[source]

Get the correpsonding Ply type string for the provided Python object.

Parameters

py_obj (object) – Python object.

Returns

Ply type string.

Return type

str

yggdrasil.metaschema.datatypes.ScalarMetaschemaType module

class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.BytesMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A bytes value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A bytes value with or without units.'
fixed_properties = {'subtype': 'bytes'}
metadata_properties = ['type', 'precision', 'units']
name = 'bytes'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'bytes'>, <class 'numpy.bytes_'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1
class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ComplexMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A complex value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A complex value with or without units.'
fixed_properties = {'subtype': 'complex'}
metadata_properties = ['type', 'precision', 'units']
name = 'complex'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'complex'>, <class 'numpy.complex128'>, <class 'numpy.complex64'>, <class 'numpy.complex128'>, <class 'numpy.complex256'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1
class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.FloatMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A float value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A float value with or without units.'
fixed_properties = {'subtype': 'float'}
metadata_properties = ['type', 'precision', 'units']
name = 'float'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'float'>, <class 'numpy.float64'>, <class 'numpy.float16'>, <class 'numpy.float32'>, <class 'numpy.float64'>, <class 'numpy.float128'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1
class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.IntMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A int value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A int value with or without units.'
fixed_properties = {'subtype': 'int'}
metadata_properties = ['type', 'precision', 'units']
name = 'int'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'int'>, <class 'numpy.int64'>, <class 'numpy.int8'>, <class 'numpy.int16'>, <class 'numpy.int32'>, <class 'numpy.int64'>, <class 'numpy.signedinteger'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1
class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType(**typedef)[source]

Bases: yggdrasil.metaschema.datatypes.MetaschemaType.MetaschemaType

Type associated with a scalar.

Developer Notes:

yggdrasil defines scalars as an umbrella type encompassing int, uint, float, bytes, and unicode.

classmethod as_python_type(obj, typedef)[source]

Convert a possible numpy type into a native Python type if possible.

Parameters
  • obj (object) – Object to convert.

  • typedef (dict) – Type definition for the object.

Returns

Native Python version of input object if conversion possible.

Return type

object

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type', 'subtype']
description = 'A scalar value with or without units.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

classmethod encode_data_readable(obj, typedef)[source]

Encode an object’s data in a readable format that may not be decoded in exactly the same way.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

extract_properties = ['type', 'title', 'subtype', 'precision', 'units']
classmethod from_array(arr, unit_str=None, dtype=None, typedef=None)[source]

Get object representation of the data.

Parameters
  • arr (np.ndarray) – Numpy array.

  • unit_str (str, optional) – Units that should be added to returned object.

  • dtype (np.dtype, optional) – Numpy data type that should be maintained as a base class when adding units. Defaults to None and is determined from the object or typedef (if provided).

  • typedef (dict, optional) – Type definition that should be used to decode the object. Defaults to None and is determined from the object or dtype (if provided).

Returns

Object representation of the data in the input array.

Return type

object

classmethod get_extract_properties(metadata)[source]

Get the list of properties that should be kept when extracting a typedef from message metadata.

Parameters

metadata (dict) – Metadata that typedef is being extracted from.

Returns

Keywords that should be kept in the typedef.

Return type

list

classmethod get_test_data(typedef=None)[source]

object: Test data.

metadata_properties = ['type', 'subtype', 'precision', 'units']
name = 'scalar'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

properties = ['type', 'title', 'subtype', 'precision', 'units']
python_types = (<class 'str'>, <class 'numpy.complex256'>, <class 'numpy.float128'>, <class 'numpy.uint64'>, <class 'numpy.int64'>, <class 'numpy.str_'>, <class 'numpy.complex128'>, <class 'complex'>, <class 'numpy.float64'>, <class 'numpy.uint32'>, <class 'numpy.int32'>, <class 'bytes'>, <class 'numpy.bytes_'>, <class 'numpy.unsignedinteger'>, <class 'numpy.complex64'>, <class 'float'>, <class 'numpy.float32'>, <class 'numpy.uint16'>, <class 'numpy.int16'>, <class 'int'>, <class 'numpy.signedinteger'>, <class 'numpy.float16'>, <class 'numpy.uint8'>, <class 'numpy.int8'>, <class 'unyt.array.unyt_quantity'>)
classmethod to_array(obj)[source]

Get np.array representation of the data.

Parameters

obj (object) – Object to get array for.

Returns

Array representation of object.

Return type

np.ndarray

classmethod transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict, optional) – Type definition that should be used to transform the object. Defaults to None.

Returns

Transformed object.

Return type

object

classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.UintMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A uint value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A uint value with or without units.'
fixed_properties = {'subtype': 'uint'}
metadata_properties = ['type', 'precision', 'units']
name = 'uint'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'numpy.uint64'>, <class 'numpy.uint8'>, <class 'numpy.uint16'>, <class 'numpy.uint32'>, <class 'numpy.uint64'>, <class 'numpy.unsignedinteger'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1
class yggdrasil.metaschema.datatypes.ScalarMetaschemaType.UnicodeMetaschemaType(**typedef)

Bases: yggdrasil.metaschema.datatypes.FixedMetaschemaType.FixedMetaschemaType, yggdrasil.metaschema.datatypes.ScalarMetaschemaType.ScalarMetaschemaType

A unicode value with or without units.

Developer Notes:

Precision X is preserved.

definition_properties = ['type']
description = 'A unicode value with or without units.'
fixed_properties = {'subtype': 'unicode'}
metadata_properties = ['type', 'precision', 'units']
name = 'unicode'
properties = ['type', 'title', 'precision', 'units']
python_types = (<class 'str'>, <class 'numpy.str_'>, <class 'unyt.array.unyt_quantity'>)
specificity = 1

yggdrasil.metaschema.datatypes.SchemaMetaschemaType module

class yggdrasil.metaschema.datatypes.SchemaMetaschemaType.SchemaMetaschemaType(*args, **kwargs)[source]

Bases: yggdrasil.metaschema.datatypes.JSONObjectMetaschemaType.JSONObjectMetaschemaType

Schema type.

classmethod decode_data(obj, typedef)[source]

Decode an object.

Parameters
  • obj (string) – Encoded object to decode.

  • typedef (dict) – Type definition that should be used to decode the object.

Returns

Decoded object.

Return type

object

definition_properties = ['type']
description = 'A schema type for evaluating subschema.'
classmethod encode_data(obj, typedef)[source]

Encode an object’s data.

Parameters
  • obj (object) – Object to encode.

  • typedef (dict) – Type definition that should be used to encode the object.

Returns

Encoded object.

Return type

string

example_data = {'type': 'boolean'}
inherit_properties = ['extract_properties']
metadata_properties = ['type']
name = 'schema'
classmethod normalize(obj)[source]

Normalize an object, if possible, to conform to this type.

Parameters

obj (object) – Object to normalize.

Returns

Normalized object.

Return type

object

properties = ['type']
specificity = 1
classmethod validate(obj, raise_errors=False)[source]

Validate an object to check if it could be of this type.

Parameters
  • obj (object) – Object to validate.

  • raise_errors (bool, optional) – If True, errors will be raised when the object fails to be validated. Defaults to False.

Returns

True if the object could be of this type, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.conversions module

yggdrasil.metaschema.datatypes.conversions.get_conversion(from_type, to_type)[source]

Get a conversion function for moving from one type to the other.

Parameters
  • from_type (str) – Name of type being converted from.

  • to_type (str) – Name of type being converted to.

Returns

Method for performing the conversion. If there is no known

conversion between the specified types, None will be returned.

Return type

function

yggdrasil.metaschema.datatypes.conversions.obj2ply(obj)[source]

Convert an Obj object to a Ply object.

Parameters

obj (dict) – Obj type object.

Returns

Ply data container.

Return type

dict

yggdrasil.metaschema.datatypes.conversions.ply2obj(ply)[source]

Convert a Ply object to an Obj object.

Parameters

ply (dict) – Ply type object.

Returns

Obj data container.

Return type

dict

yggdrasil.metaschema.datatypes.conversions.register_conversion(from_type, to_type)[source]

Register a conversion function for going between types.

Parameters
  • from_type (str) – Name of type being converted from.

  • to_type (str) – Name of type being converted to.

Returns

Decorator that will register the conversion.

Return type

function

Module contents

class yggdrasil.metaschema.datatypes.MetaschemaTypeMeta(name, bases, class_dict)[source]

Bases: type

Meta class for registering datatypes.

yggdrasil.metaschema.datatypes.add_type_from_schema(path_to_schema, **kwargs)[source]

Add a type from a schema in a file.

Parameters
  • path_to_schema (string) – Full path to the location of a schema file that can be loaded.

  • target_globals (dict, optional) – Globals dictionary for module where the fixed class should be added. If None, the new class is returned. Defaults to local globals.

  • **kwargs – Additional keyword arguments are assumed to be attributes for the new class.

yggdrasil.metaschema.datatypes.compare_schema(schema1, schema2, root1=None, root2=None)[source]

Compare two schemas for compatibility.

Parameters
  • schema1 (dict) – First schema.

  • schema2 (dict) – Second schema.

  • root1 (dict, optional) – Root for first schema. Defaults to None and is set to schema1.

  • root2 (dict, optional) – Root for second schema. Defaults to None and is set to schema2.

Yields

str – Comparision failure messages.

yggdrasil.metaschema.datatypes.complete_typedef(typedef)[source]

Complete the type definition by converting it into the standard format.

Parameters

typedef (str, dict, list) – A type name, type definition dictionary, dictionary of subtype definitions, or a list of subtype definitions.

Returns

Type definition dictionary.

Return type

dict

Raises

TypeError – If typedef is not a valid type.

yggdrasil.metaschema.datatypes.decode(msg)[source]

Decode an object from a message.

Parameters

msg (bytes) – Bytes encoded message.

Returns

Decoded Python object.

Return type

object

yggdrasil.metaschema.datatypes.decode_data(obj, typedef)[source]

Decode a JSON serializable object to get the corresponding Python variable.

Parameters
  • obj (object) – JSON serializable object representing an encoded form of a message.

  • typedef (dict) – JSON schema describing the object.

Returns

Deserialized version of the object.

Return type

object

yggdrasil.metaschema.datatypes.encode(obj)[source]

Encode an object into a message.

Parameters

obj (object) – Python object to be encoded.

Returns

Encoded message.

Return type

bytes

yggdrasil.metaschema.datatypes.encode_data(obj, typedef=None)[source]

Encode an object into a JSON serializable object.

Parameters
  • obj (object) – Python object to be encoded.

  • typedef (dict, optional) – JSON schema describing the object. Defaults to None and class is determined from the object.

Returns

JSON serializable version of the object.

Return type

object

yggdrasil.metaschema.datatypes.encode_data_readable(obj, typedef=None)[source]

Encode an object into a JSON serializable object that is human readable but dosn’t guarantee identical deserialization.

Parameters
  • obj (object) – Python object to be encoded.

  • typedef (dict, optional) – JSON schema describing the object. Defaults to None and class is determined from the object.

Returns

JSON serializable version of the object.

Return type

object

yggdrasil.metaschema.datatypes.encode_type(obj, typedef=None, **kwargs)[source]

Encode an object into a JSON schema that can be used to both describe the object and validate others.

Parameters
  • obj (object) – Python object to be encoded.

  • typedef (dict, optional) – Type properties that should be used to initialize the encoded type definition in certain cases. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are passed to the identified type class’s encode_type method.

Returns

Encoded JSON schema describing the object.

Return type

dict

yggdrasil.metaschema.datatypes.generate_data(typedef, **kwargs)[source]

Generate mock data for the specified type.

Parameters
  • typedef (dict) – Type definition.

  • **kwargs – Additional keyword arguments are passed to the generating class’s generate_data method.

Returns

Python object of the specified type.

Return type

object

yggdrasil.metaschema.datatypes.get_registered_types()[source]

Return a dictionary of registered types.

Returns

Registered type/class pairs.

Return type

dict

yggdrasil.metaschema.datatypes.get_type_class(type_name)[source]

Return a type class given it’s name.

Parameters

type_name (str, list) – Name of type class or list of names of type classes.

Returns

Type class.

Return type

class

yggdrasil.metaschema.datatypes.get_type_from_def(typedef, dont_complete=False)[source]

Return the type instance based on the provided type definition.

Parameters
  • typedef (obj) – This can be the name of a type, a dictionary containing a type definition (the ‘typename’ keyword must be specified), or a complex type (a list or dictionary containing types).

  • dont_complete (bool, optional) – If True, the type definition will be used as-is. Otherwise it will be completed using normalization which can be time consuming. Defaults to False.

Returns

Instance of the appropriate type class.

Return type

MetaschemaType

yggdrasil.metaschema.datatypes.guess_type_from_msg(msg)[source]

Guess the type class from a message.

Parameters

msg (str, bytes) – Message containing metadata.

Raises

ValueError – If a type class cannot be determined.

Returns

Instance of the appropriate type class.

Return type

MetaschemaType

yggdrasil.metaschema.datatypes.guess_type_from_obj(obj)[source]

Guess the type class for a given Python object.

Parameters

obj (object) – Python object.

Returns

Instance of the appropriate type class.

Return type

MetaschemaType

Raises

ValueError – If a type class cannot be determined.

yggdrasil.metaschema.datatypes.import_schema_types()[source]

Import all types to ensure they are registered.

yggdrasil.metaschema.datatypes.is_default_typedef(typedef)[source]

Determine if a type definition is the default type definition.

Parameters

typedef (dict) – Type definition to test.

Returns

True if typedef is the default, False otherwise.

Return type

bool

yggdrasil.metaschema.datatypes.register_type(type_class)[source]

Register a type class, recording methods for encoding/decoding.

Parameters

type_class (class) – Class to be registered.

Raises
  • ValueError – If the type is already registered.

  • ValueError – If the type is a default JSON type.

  • ValueError – If any of the type’s properties are not registered.

yggdrasil.metaschema.datatypes.resolve_schema_references(schema, resolver=None)[source]

Resolve references within a schema.

Parameters
  • schema (dict) – Schema with references to resolve.

  • top_level (dict, optional) – Reference to the top level schema.

Returns

Schema with references replaced with internal references.

Return type

dict

yggdrasil.metaschema.datatypes.transform_type(obj, typedef=None)[source]

Transform an object based on type info.

Parameters
  • obj (object) – Object to transform.

  • typedef (dict) – Type definition that should be used to transform the object.

Returns

Transformed object.

Return type

object

yggdrasil.drivers package

Submodules

yggdrasil.drivers.BuildModelDriver module

class yggdrasil.drivers.BuildModelDriver.BuildModelDriver(*args, **kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.CompiledModelDriver

Class for running build file compiled drivers.

Parameters
  • name (str) – Driver name.

  • args (str, list) – Executable that should be created (make target) and any arguments for the executable.

  • target (str, optional) – Make target that should be built to create the model executable. Defaults to None.

  • target_language (str, optional) – Language that the target is written in. Defaults to None and will be set based on the source files provided.

  • target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.

  • target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.

  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • env_compiler (str, optional) – Environment variable where the compiler executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.

  • env_compiler_flags (str, optional) – Environment variable where the compiler flags should be stored (including those required to compile against the yggdrasil interface). If not provided, this will be determined by the target language driver.

  • env_linker (str, optional) – Environment variable where the linker executable should be stored for use within the Makefile. If not provided, this will be determined by the target language driver.

  • env_linker_flags (str, optional) – Environment variable where the linker flags should be stored (including those required to link against the yggdrasil interface). If not provided, this will be determined by the target language driver.

  • **kwargs – Additional keyword arguments are passed to parent class.

buildfile

Path to file containing build instructions.

Type

str

builddir

Path to directory where build products will be saved.

Type

str

sourcedir

Path to directory where source files are located.

Type

str

compiledir

Path to directory where compilation call should be made from. Defaults to working_dir.

Type

str

target

Name of executable that should be created and called.

Type

str

target_language

Language that the target is written in.

Type

str

target_language_driver

Language driver for the target language.

Type

ModelDriver

target_compiler

Compilation tool that should be used to compile the target language.

Type

str

target_linker

Compilation tool that should be used to link the target language.

Type

str

target_compiler_flags

Compilation flags that should be passed to the target language compiler.

Type

list

target_linker_flags

Linking flags that should be passed to the target language linker.

Type

list

env_compiler

Compiler environment variable.

Type

str

env_compiler_flags

Compiler flags environment variable.

Type

str

env_linker

Linker environment variable.

Type

str

env_linker_flags

Linker flags environment variable.

Type

str

Class Attributes:
built_where_called (bool): If True, it is assumed that compilation

output will be saved in the same directory from which the compilation command is issued.

Raises

RuntimeError – If neither the IPC or ZMQ C libraries are available.

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration. For compiled languages this includes selecting the default compiler. The order of precedence is the config file ‘compiler’ option for the language, followed by the environment variable set by _compiler_env, followed by the existing class attribute.

allow_parallel_build = False
base_languages = ['c']
buildfile_base = None
built_where_called = False
classmethod call_compiler(src, **kwargs)[source]

Compile a source file into an executable or linkable object file, checking for errors.

Parameters
  • src (str) – Full path to source file.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Full path to compiled source.

Return type

str

cleanup()[source]

Remove compiled executable.

compile_dependencies_instance(*args, **kwargs)[source]

Compile dependencies specifically for this instance.

compile_model(**kwargs)[source]

Compile model executable(s).

Parameters

**kwargs – Keyword arguments are passed on to the parent class’s method.

executable_type = 'build'
classmethod fix_path(path, for_env=False)[source]

Update a path.

Parameters
  • path (str) – Path that should be formatted.

  • for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.

Returns

Updated path.

Return type

str

full_language = False
classmethod get_buildfile_lock(**kwargs)[source]

Get a lock for a buildfile to prevent simultaneous access, creating one as necessary.

classmethod get_language_for_buildfile(buildfile, target=None)[source]

Determine the target language based on the contents of a build file.

Parameters
  • buildfile (str) – Full path to the build configuration file.

  • target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_language_for_source(fname=None, buildfile=None, languages=None, early_exit=False, call_base=False, **kwargs)[source]

Determine the language that can be used with the provided source file(s). If more than one language applies to a set of multiple files, the language that applies to the most files is returned.

Parameters
  • fname (str, list) – The full path to one or more files. If more than one is provided, they are iterated over.

  • buildfile (str, optional) – Full path to the build configuration file. Defaults to None and will be searched for.

  • languages (list, optional) – The list of languages that are acceptable. Defaults to None and any language will be acceptable.

  • early_exit (bool, optional) – If True, the first language identified will be returned if fname is a list of files. Defaults to False.

  • source_dir (str, optional) – Full path to the directory containing the source files. Defaults to None and is determiend from fname.

  • buildfile – Full path to the build configuration file. Defaults to None and will be searched for.

  • target (str, optional) – The build target. Defaults to None.

  • call_base (bool, optional) – If True, the base class’s method is called directly. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The language that can operate on the specified file.

Return type

str

classmethod get_source_dir(fname=None, source_dir=None)[source]
classmethod get_target_language_info(target_language_driver=None, target_language='c', target_compiler=None, target_compiler_flags=None, target_linker=None, target_linker_flags=None, logging_level=None, without_wrapper=False, compiler_flag_kwargs=None, linker_flag_kwargs=None, **kwargs)[source]

Get a dictionary of information about language compilation tools.

Parameters
  • target_language_driver (ModelDriver, optional) – Driver associated with the target language. If not provided, one will be created based on ‘target_language’.

  • target_language (str, optional) – Language to get info for. Defaults to ‘c’.

  • target_compiler (str, optional) – Compilation tool that should be used to compile the target language. Defaults to None and will be set based on the selected language driver.

  • target_linker (str, optional) – Compilation tool that should be used to link the target language. Defaults to None and will be set based on the selected language driver.

  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • logging_level (int, optional) – The numeric logging level that should be passed as a definition. Defaults to None and is ignored.

  • without_wrapper (bool, optional) – If True, the returned info will be updated for compilation when a build wrapper was not created. Defaults to False.

  • compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.

  • linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.

  • **kwargs – Additional keyword arguments are added to the output dictionary.

Returns

Information about language compilers and linkers.

Return type

dict

is_build_tool = True
classmethod is_source_file(fname)[source]

Determine if the provided file name points to a source files for the associated programming language by checking the extension.

Parameters

fname (str) – Path to file.

Returns

True if the provided file is a source file, False otherwise.

Return type

bool

parse_arguments(args, **kwargs)[source]

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters
  • args (list) – List of arguments provided.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

set_env(**kwargs)[source]

Get environment variables that should be set for the model process.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

classmethod set_env_compiler(language_info=None, **kwargs)[source]

Get environment variables that should be set for the compilation process.

Parameters
  • language_info (dict) – Language compilation tool information.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

set_target_language()[source]

Set the language of the target being compiled (usually the same as the language associated with this driver.

Returns

Name of language.

Return type

str

sourcedir_as_sourcefile = False
property target_language_info

Information about the underlying language.

Type

dict

class yggdrasil.drivers.BuildModelDriver.BuildToolBase(**kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.CompilerBase

Base class for build tools.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = None
classmethod get_flags(**kwargs)[source]

Get compilation flags, replacing outfile with target.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

classmethod get_tool_suffix()[source]

Get the string that should be added to tool products based on the tool used.

Returns

Suffix that should be added to tool products to indicate the

tool used.

Return type

str

is_build_tool = True

yggdrasil.drivers.CMakeModelDriver module

class yggdrasil.drivers.CMakeModelDriver.CMakeBuilder(**kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.LinkerBase

CMake build tool.

classmethod call(*args, **kwargs)[source]

Print contents of CMakeCache.txt before raising error.

default_executable = 'cmake'
default_flags = []
executable_ext = ''
classmethod extract_kwargs(kwargs, **kwargs_ex)[source]

Extract linker kwargs, leaving behind just compiler kwargs.

Parameters
  • kwargs (dict) – Keyword arguments passed to the compiler that should be sorted into kwargs used by either the compiler or linker or both. Keywords that are not used by the compiler will be removed from this dictionary.

  • **kwargs_ex – Additional keyword arguments are passed to the parent class’s method.

Returns

Keyword arguments that should be passed to the linker.

Return type

dict

flag_options = {'builddir': {'key': '--build', 'position': 0}, 'configuration': '--config', 'target': '--target'}
classmethod get_executable_command(args, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(target=None, builddir=None, **kwargs)[source]

Get a list of build flags for building a project using cmake.

Parameters
  • target (str, optional) – Target that should be built. Defaults to to None and is ignored.

  • builddir (str, optional) – Directory containing the build tree. Defaults to None and is set based on outfile is provided or cls.default_builddir if not. Defaults to ‘.’ (which will be the current working directory).

  • **kwargs – Additional keyword arguments are ignored.

Returns

Linker flags.

Return type

list

classmethod get_output_file(obj, target=None, builddir=None, **kwargs)[source]

Determine the appropriate output file that will result when bulding a given directory.

Parameters
  • obj (str) – Directory being built or a file in the directory being built.

  • target (str, optional) – Target that will be used to create the output file. Defaults to None. Target is required in order to determine the name of the file that will be created.

  • builddir (str, optional) – Directory where build tree should be created. Defaults to None and obj will used (if its a directory) or the directory containing obj will be used (if its a file).

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Full path to file that will be produced.

Return type

str

Raises

RuntimeError – If target is None.

is_gnu = False
languages = ['cmake']
library_ext = '.so'
output_key = None
tool_suffix_format = ''
toolname = 'cmake'
class yggdrasil.drivers.CMakeModelDriver.CMakeConfigure(**kwargs)[source]

Bases: yggdrasil.drivers.BuildModelDriver.BuildToolBase

CMake configuration tool.

add_libraries = False
classmethod append_product(products, src, new, **kwargs)[source]

Append a product to the specified list along with additional values indicated by cls.product_exts.

Parameters
  • products (list) – List of of existing products that new product should be appended to.

  • src (list) – Input arguments to compilation call that was used to generate the output file (usually one or more source files).

  • new (str) – New product that should be appended to the list.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

build_language = 'cmake'
classmethod call(args, **kwargs)[source]

Call the tool with the provided arguments. If the first argument resembles the name of the tool executable, the executable will not be added.

Parameters
  • args (list) – The arguments that should be passed to the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method and the associated linker/archiver’s call method if dont_link is False.

Returns

Output to stdout from the command execution if skip_flags is

True, produced file otherwise.

Return type

str

combine_with_linker = False
compile_only_flag = None
classmethod create_include(fname, target, compiler=None, compiler_flags=None, linker=None, linker_flags=None, library_flags=None, internal_library_flags=None, configuration='Release', verbose=False, **kwargs)[source]

Create CMakeList include file with necessary includes, definitions, and linker flags.

Parameters
  • fname (str) – File where the include file should be saved.

  • target (str) – Target that links should be added to.

  • compiler (CompilerBase) – Compiler that should be used to generate the list of compilation flags.

  • compile_flags (list, optional) – Additional compile flags that should be set. Defaults to [].

  • linker (LinkerBase) – Linker that should be used to generate the list of compilation flags.

  • linker_flags (list, optional) – Additional linker flags that should be set. Defaults to [].

  • library_flags (list, optional) – List of library flags to add. Defaults to [].

  • internal_library_flags (list, optional) – List of library flags associated with yggdrasil libraries. Defaults to [].

  • configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’. Only used on Windows to determin the standard library.

  • verbose (bool, optional) – If True, the contents of the created file are displayed. Defaults to False.

  • **kwargs – Additional keyword arguments are ignored.

Returns

Lines that should be added before the executable is defined

in the CMakeLists.txt (e.g. LINK_DIRECTORIES commands).

Return type

list

Raises

ValueError – If a linker or compiler flag cannot be interpreted.

default_archiver = False
default_builddir = '.'
default_executable = 'cmake'
default_flags = []
classmethod fix_path(x, is_gnu=False)[source]

Fix paths so that they conform to the format expected by the OS and/or build tool.

flag_options = {'builddir': '-B%s', 'configuration': '-DCMAKE_BUILD_TYPE=%s', 'definitions': '-D%s', 'generator': '-G%s', 'platform': '-A%s', 'sourcedir': '', 'toolset': '-T%s'}
classmethod generator(return_default=False, default=None, **kwargs)[source]

Determine the generator that should be used.

Parameters
  • return_default (bool, optional) – If True, the default generator will be returned even if the environment variable is set. Defaults to False.

  • default (str, optional) – Value that should be returned if a generator cannot be located. Defaults to None.

  • **kwargs – Keyword arguments are passed to cls.call.

Returns

Name of the generator.

Return type

str

classmethod generator2toolset(generator)[source]

Determine the toolset string option that corresponds to the provided generator name.

Parameters

generator (str) – Name of the generator.

Returns

Name of the toolset.

Return type

str

Raises
  • NotImplementedError – If the platform is not windows.

  • ValueError – If the generator is not a flavor of Visual Studio.

  • ValueError – If a tool set cannot be located for the specified generator.

classmethod get_executable_command(args, **kwargs)[source]

Determine the command required to run the tool using the specified arguments and options.

Parameters
  • args (list) – The arguments that should be passed to the tool. If skip_flags is False, these are treated as input files that will be used by the tool.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Output to stdout from the command execution.

Return type

str

classmethod get_flags(sourcedir='.', builddir=None, target_compiler=None, target_linker=None, **kwargs)[source]

Get a list of configuration/generation flags.

Parameters
  • sourcedir (str, optional) – Directory containing the source files to be compiled and the target CMakeLists.txt file. Defaults to ‘.’ (the current working directory).

  • builddir (str, optional) – Directory that will contain the build tree. Defaults to ‘.’ (this current working directory).

  • target_compiler (str, optional) – Compiler that should be used by cmake. Defaults to None and the default for the target language will be used.

  • target_linker (str, optional) – Linker that should be used by cmake. Defaults to None and the default for the target language will be used.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Compiler flags.

Return type

list

Raises
  • RuntimeError – If dont_link is True and the provide outfile and builddir keyword arguments point to conflicting paths.

  • ValueError – If ‘include_dirs’ is set (‘sourcedir’ should be used for cmake to specify the location of the source).

classmethod get_output_file(src, dont_link=False, dont_build=None, sourcedir=None, builddir=None, working_dir=None, **kwargs)[source]

Determine the appropriate output file or directory that will result when configuring/building a given source directory.

Parameters
  • src (str) – Directory containing source files being compiled.

  • dont_link (bool, optional) – If True, the result assumes that the source is just compiled and not linked. If False, the result will be the final result after linking.

  • dont_build (bool, optional) – Alias for dont_link. If not None, this keyword overrides the value of dont_link. Defaults to None.

  • sourcedir (str, optional) – Directory where sources files are located. Defaults to None. If None, src will be used to determine the value.

  • builddir (str, optional) – Directory where build tree should be created. Defaults to None. If None, sourcedir will be used.

  • working_dir (str, optional) – Working directory where output file should be located. Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are ignored unless dont_link is False; then they are passed to get_linker_output_file

Returns

Full path to file that will be produced.

Return type

str

is_gnu = False
is_linker = False
languages = ['cmake']
output_key = None
product_files = ['Makefile', 'CMakeCache.txt', 'cmake_install.cmake', 'CMakeFiles']
remove_product_exts = ['CMakeFiles']
toolname = 'cmake'
class yggdrasil.drivers.CMakeModelDriver.CMakeModelDriver(*args, **kwargs)[source]

Bases: yggdrasil.drivers.BuildModelDriver.BuildModelDriver

Class for running cmake compiled drivers. Before running the cmake command, the cmake commands for setting the necessary compiler & linker flags for the interface’s C/C++ library are written to a file called ‘ygg_cmake.txt’ that should be included in the CMakeLists.txt file (after the target executable has been added).

Parameters
  • name (str) – Driver name.

  • args (str, list) – Executable that should be created (cmake target) and any arguments for the executable.

  • sourcedir (str, optional) – Source directory to call cmake on. If not provided it is set to working_dir. This should be the directory containing the CMakeLists.txt file. It can be relative to working_dir or absolute.

  • builddir (str, optional) – Directory where the build should be saved. Defaults to <sourcedir>/build. It can be relative to working_dir or absolute.

  • configuration (str, optional) – Build type/configuration that should be built. Defaults to ‘Release’.

  • **kwargs – Additional keyword arguments are passed to parent class.

sourcedir

Source directory to call cmake on.

Type

str

add_libraries

If True, interface libraries and dependency libraries are added using CMake’s ADD_LIBRARY directive. If False, interface libraries are found using FIND_LIBRARY.

Type

bool

configuration

Build type/configuration that should be built. This is only used on Windows.

Type

str

Raises

RuntimeError – If neither the IPC or ZMQ C libraries are available.

add_libraries = False
buildfile_base = 'CMakeLists.txt'
property buildfile_orig

Full path to where the original CMakeLists.txt file will be stored during compilation of the modified file.

Type

str

property buildfile_ygg

Full path to the verison of the CMakeLists.txt that has been updated w/ yggdrasil compilation flags.

Type

str

cfg = <yggdrasil.config.YggConfigParser object>
compile_model(target=None, **kwargs)[source]

Compile model executable(s) and appends any products produced by the compilation that should be removed after the run is complete.

Parameters
  • target (str, optional) – Target to build.

  • **kwargs – Keyword arguments are passed on to the call_compiler method.

default_archiver = None
default_archiver_flags = None
default_compiler = 'cmake'
default_compiler_flags = None
default_linker = 'cmake'
default_linker_flags = None
classmethod fix_path(path, for_env=False, **kwargs)[source]

Update a path.

Parameters
  • path (str) – Path that should be formatted.

  • for_env (bool, optional) – If True, the path is formatted for use in and environment variable. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Updated path.

Return type

str

classmethod get_language_for_buildfile(buildfile, target=None)[source]

Determine the target language based on the contents of a build file.

Parameters
  • buildfile (str) – Full path to the build configuration file.

  • target (str, optional) – Target that will be built. Defaults to None and the default target in the build file will be used.

classmethod get_target_language_info(target_compiler_flags=None, target_linker_flags=None, compiler_flag_kwargs=None, linker_flag_kwargs=None, without_wrapper=False, **kwargs)[source]

Get a dictionary of information about language compilation tools.

Parameters
  • target_compiler_flags (list, optional) – Compilation flags that should be passed to the target language compiler. Defaults to [].

  • target_linker_flags (list, optional) – Linking flags that should be passed to the target language linker. Defaults to [].

  • compiler_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_compiler_flags method. Defaults to None.

  • linker_flag_kwargs (dict, optional) – Keyword arguments to pass to the get_linker_flags method. Defaults to None.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Information about language compilers and linkers.

Return type

dict

inverse_type_map = None
language = 'cmake'
parse_arguments(args, **kwargs)[source]

Sort arguments based on their syntax to determine if an argument is a source file, compilation flag, or runtime option/flag that should be passed to the model executable.

Parameters
  • args (list) – List of arguments provided.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

classmethod prune_sh_gcc(path, gcc)[source]

Remove instances of sh.exe from the path that are not associated with the selected gcc compiler. This can happen on windows when rtools or git install a version of sh.exe that is added to the path before the compiler.

Parameters
  • path (str) – Contents of the path variable.

  • gcc (str) – Full path to the gcc executable.

Returns

Modified path that removes the extra instances

of sh.exe.

Return type

str

sourcedir_as_sourcefile = True
classmethod update_compiler_kwargs(**kwargs)[source]

Update keyword arguments supplied to the compiler get_flags method for various options.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Keyword arguments for a get_flags method providing compiler

flags.

Return type

dict

use_env_vars = False
write_wrappers(**kwargs)[source]

Write any wrappers needed to compile and/or run a model.

Parameters

**kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Full paths to any created wrappers.

Return type

list

yggdrasil.drivers.CModelDriver module

class yggdrasil.drivers.CModelDriver.ARArchiver(**kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.ArchiverBase

Archiver class for ar tool.

compatible_toolsets = ['gnu', 'llvm']
default_executable = 'ar'
default_executable_env = 'AR'
default_flags = []
default_flags_env = None
is_gnu = True
languages = ['c', 'c++', 'fortran']
library_ext = '.a'
output_first_library = True
output_key = ''
search_path_envvar = ['LIBRARY_PATH']
shared_library_flag = None
static_library_flag = 'rcs'
toolname = 'ar'
toolset = 'gnu'
class yggdrasil.drivers.CModelDriver.CCompilerBase(**kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.CompilerBase

Base class for C compilers.

static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

classmethod call(args, **kwargs)[source]

Call the compiler with the provided arguments. For yggdrasil C models will always be linked using the C++ linker since some parts of the interface library are written in C++.

default_executable_env = 'CC'
default_flags = ['-g', '-Wall']
default_flags_env = 'CFLAGS'
classmethod get_search_path(*args, **kwargs)[source]

Determine the paths searched by the tool for external library files.

Parameters
  • *args – Additional arguments are passed to the parent class’s method.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

List of paths that the tools will search.

Return type

list

languages = ['c']
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH']}
search_path_envvar = ['C_INCLUDE_PATH']
search_path_flags = ['-E', '-v', '-xc', '/dev/null']
search_regex = ['(?:#include <...> search starts here:)|(?: ([^\\n]+?)(?: \\(framework directory\\))?)\\n']
search_regex_begin = '#include "..." search starts here:'
search_regex_end = 'End of search list.'
classmethod set_env(*args, **kwargs)[source]

Set environment variables required for compilation.

Parameters
  • *args – Arguments are passed to the parent class’s method.

  • **kwargs – Keyword arguments are passed to the parent class’s method.

Returns

Environment variables for the model process.

Return type

dict

class yggdrasil.drivers.CModelDriver.CModelDriver(*args, **kwargs)[source]

Bases: yggdrasil.drivers.CompiledModelDriver.CompiledModelDriver

Class for running C models.

static after_registration(cls, **kwargs)[source]

Operations that should be performed to modify class attributes after registration.

classmethod allows_realloc(var)[source]

Determine if a variable allows the receive call to perform realloc.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if the variable allows realloc, False otherwise.

Return type

bool

brackets = ('{', '}')
classmethod call_linker(obj, language=None, **kwargs)[source]

Link several object files to create an executable or library (shared or static), checking for errors.

Parameters
  • obj (list) – Object files that should be linked.

  • language (str, optional) – Language that should be used to link the files. Defaults to None and the language of the current driver is used.

  • **kwargs – Additional keyword arguments are passed to run_executable.

Returns

Full path to compiled source.

Return type

str

cfg = <yggdrasil.config.YggConfigParser object>
classmethod configure(cfg, macos_sdkroot=None, vcpkg_dir=None, **kwargs)[source]

Add configuration options for this language. This includes locating any required external libraries and setting option defaults.

Parameters
  • cfg (YggConfigParser) – Config class that options should be set for.

  • macos_sdkroot (str, optional) – Full path to the root directory for the MacOS SDK that should be used. Defaults to None and is ignored.

  • vcpkg_dir (str, optional) – Full path to the root directory containing a vcpkg installation. This should be the directory that contains the vcpkg executable and any packages installed by vcpkg (in subdirectories). Defaults to None and is ignored.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Section, option, description tuples for options that could not

be set.

Return type

list

default_archiver = 'ar'
default_archiver_flags = None
default_compiler = 'gcc'
default_compiler_flags = None
default_linker = 'gcc'
default_linker_flags = None
external_libraries = {'czmq': {'include': 'czmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libczmq.so'}, 'numpy': {'include': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/numpy/core/include/numpy/arrayobject.h', 'language': 'c', 'libtype': 'header_only'}, 'python': {'include': '/opt/hostedtoolcache/Python/3.8.12/x64/include/python3.8/Python.h', 'language': 'c', 'libtype': 'shared', 'shared': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/libpython3.8.so'}, 'rapidjson': {'include': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/rapidjson/include/rapidjson/rapidjson.h', 'language': 'c', 'libtype': 'header_only'}, 'zmq': {'include': 'zmq.h', 'language': 'c', 'libtype': 'shared', 'shared': '/lib/x86_64-linux-gnu/libzmq.so'}}
classmethod finalize_function_io(direction, x)[source]

Finalize info for an input/output channel following function parsing.

Parameters
  • direction (str) – Direction of channel (‘input’ or ‘output’)

  • x (dict) – Channel info.

classmethod format_function_param(key, default=None, **kwargs)[source]

Return the formatted version of the specified key.

Parameters
  • key (str) – Key in cls.function_param mapping that should be formatted.

  • default (str, optional) – Format that should be returned if key is not in cls.function_param. Defaults to None.

  • **kwargs – Additional keyword arguments are used in formatting the request function parameter.

Returns

Formatted string.

Return type

str

Raises

NotImplementedError – If key is not in cls.function_param and default is not set.

function_param = {'and': '&&', 'assign': '{name} = {value};', 'assign_copy': 'memcpy({name}, {value}, {N}*sizeof({native_type}));', 'block_end': '}', 'break': 'break;', 'comment': '//', 'copy_any': '{name} = copy_generic({value});', 'copy_array': '{name} = copy_json_array({value});', 'copy_class': '{name} = copy_python({value});', 'copy_function': '{name} = copy_python({value});', 'copy_instance': '{name} = copy_generic({value});', 'copy_obj': '{name} = copy_obj({value});', 'copy_object': '{name} = copy_json_object({value});', 'copy_ply': '{name} = copy_ply({value});', 'copy_schema': '{name} = copy_schema({value});', 'declare': '{type_name} {variable};', 'error': 'printf("{error_msg}\\n"); return -1;', 'exec_begin': 'int main() {', 'exec_end': '  return 0;\n}', 'exec_prefix': '#include <stdbool.h>', 'false': 'false', 'false_flag': '0', 'flag_cond': '{flag_var} >= 0', 'for_begin': 'for ({iter_var} = {iter_begin}; {iter_var} < {iter_end}; {iter_var}++) {{', 'fprintf': 'printf("{message}\\n", {variables});', 'free': 'if ({variable} != NULL) {{ free({variable}); {variable} = NULL; }}', 'free_any': 'free_generic({variable});', 'free_array': 'free_json_array({variable});', 'free_class': 'destroy_python({variable});', 'free_function': 'destroy_python({variable});', 'free_instance': 'free_generic({variable});', 'free_obj': 'free_obj({variable});', 'free_object': 'free_json_object({variable});', 'free_ply': 'free_ply({variable});', 'free_schema': 'free_schema({variable});', 'function_def_begin': '{output_type} {function_name}({input_var}) {{', 'function_def_regex': '(?P<flag_type>.+?)\\s*{function_name}\\s*\\((?P<inputs>(?:[^{{])*?)\\)\\s*\\{{(?P<body>(?:.*?\\n?)*?)(?:(?:return *(?P<flag_var>.+?)?;(?:.*?\\n?)*?\\}})|(?:\\}}))', 'if_begin': 'if ({cond}) {{', 'if_elif': '}} else if ({cond}) {{', 'if_else': '}} else {{', 'import': '#include "{filename}"', 'indent': '  ', 'index': '{variable}[{index}]', 'init_any': 'init_generic()', 'init_array': 'init_json_array()', 'init_class': 'init_python()', 'init_function': 'init_python()', 'init_instance': 'init_generic()', 'init_obj': 'init_obj()', 'init_object': 'init_json_object()', 'init_ply': 'init_ply()', 'init_schema': 'init_schema()', 'init_type_1darray': 'create_dtype_1darray("{subtype}", {precision}, {length}, "{units}", {use_generic})', 'init_type_array': 'create_dtype_json_array({nitems}, {items}, {use_generic})', 'init_type_default': 'create_dtype_default("{type}", {use_generic})', 'init_type_empty': 'create_dtype_empty({use_generic})', 'init_type_ndarray': 'create_dtype_ndarray("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_ndarray_arr': 'create_dtype_ndarray_arr("{subtype}", {precision}, {ndim}, {shape}, "{units}", {use_generic})', 'init_type_obj': 'create_dtype_obj({use_generic})', 'init_type_object': 'create_dtype_json_object({nitems}, {keys}, {values}, {use_generic})', 'init_type_ply': 'create_dtype_ply({use_generic})', 'init_type_pyobj': 'create_dtype_pyobj("{type}", {use_generic})', 'init_type_scalar': 'create_dtype_scalar("{subtype}", {precision}, "{units}", {use_generic})', 'init_type_schema': 'create_dtype_schema({use_generic})', 'input': '{channel} = yggInputType("{channel_name}", {channel_type});', 'inputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(\\((?P<name_ptr>\\*+)?)?(?P<name>.+?)(?(4)(?:\\)))(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'interface': '#include "{interface_library}"', 'line_end': ';', 'not': '!', 'not_flag_cond': '{flag_var} < 0', 'null': 'NULL', 'output': '{channel} = yggOutputType("{channel_name}", {channel_type});', 'outputs_def_regex': '\\s*(?P<native_type>(?:[^\\s\\*])+(\\s+)?(?P<ptr>\\*+)?)(?(ptr)(?(1)(?:\\s*)|(?:\\s+)))(?P<name>.+?)(?P<shape>(?:\\[.+?\\])+)?\\s*(?:,|$)(?:\\n)?', 'print': 'printf("{message}\\n");', 'print_any': 'display_generic({object});', 'print_array': 'display_json_array({object});', 'print_bytes': 'printf("%s\\n", {object});', 'print_class': 'display_python({object});', 'print_complex': 'print_complex({object});', 'print_float': 'printf("%f\\n", {object});', 'print_function': 'display_python({object});', 'print_instance': 'display_generic({object});', 'print_int': 'printf("%i\\n", {object});', 'print_obj': 'display_obj({object});', 'print_object': 'display_json_object({object});', 'print_ply': 'display_ply({object});', 'print_schema': 'display_schema({object});', 'print_string': 'printf("%s\\n", {object});', 'print_uint': 'printf("%u\\n", {object});', 'print_unicode': 'printf("%s\\n", {object});', 'quote': '"', 'recv_function': 'yggRecvRealloc', 'recv_heap': 'yggRecvRealloc', 'recv_stack': 'yggRecv', 'return': 'return {output_var};', 'send_function': 'yggSend', 'true': 'true', 'true_flag': '1', 'while_begin': 'while ({cond}) {{'}
classmethod get_dependency_info(dep, toolname=None, default=None)[source]

Get the dictionary of information associated with a dependency.

Parameters
  • dep (str) – Name of internal or external dependency or full path to the library.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • default (dict, optional) – Information dictionary that should be returned if dep cannot be located. Defaults to None and an error will be raised if dep cannot be found.

Returns

Dependency info.

Return type

dict

classmethod get_json_type(native_type)[source]

Get the JSON type from the native language type.

Parameters

native_type (str) – The native language type.

Returns

The JSON type.

Return type

str, dict

classmethod get_name_declare(var)[source]

Determine the name that should be used for declaration.

Parameters

var (str, dict) – Name of variable or dictionary of information.

Returns

Modified name for declaration.

Return type

str

classmethod get_native_type(**kwargs)[source]

Get the native type.

Parameters
  • type (str, optional) – Name of yggdrasil extended JSON type or JSONSchema dictionary defining a datatype.

  • **kwargs – Additional keyword arguments may be used in determining the precise declaration that should be used.

Returns

The native type.

Return type

str

classmethod get_testing_options(**kwargs)[source]

Method to return a dictionary of testing options for this class.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class.

Returns

Dictionary of variables to use for testing. Key/value pairs:

kwargs (dict): Keyword arguments for driver instance. deps (list): Dependencies to install.

Return type

dict

include_channel_obj = True
classmethod input2output(var)[source]

Perform conversion necessary to turn a variable extracted from a function definition from an input to an output.

Parameters

var (dict) – Variable definition.

Returns

Updated variable definition.

Return type

dict

interface_dependencies = ['rapidjson']
interface_directories = ['/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C']
interface_library = 'ygg'
interface_map = {'call': 'flag = rpcCall({channel_obj}, {outputs}, {inputs})', 'client': 'yggRpcClientType("{channel_name}", {datatype_out}, {datatype_in})', 'import': '#include "YggInterface.h"', 'input': 'yggInputType("{channel_name}", {datatype})', 'output': 'yggOutputType("{channel_name}", {datatype})', 'recv': 'flag = yggRecv({channel_obj}, {input_refs})', 'send': 'flag = yggSend({channel_obj}, {outputs})', 'server': 'yggRpcServerType("{channel_name}", {datatype_in}, {datatype_out})', 'timesync': 'yggTimesync("{channel_name}", "{time_units}")'}
internal_libraries = {'datatypes': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/datatypes', 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': [], 'internal_dependencies': ['regex'], 'language': 'c++', 'libtype': 'object'}, 'python_wrapper': {'compiler_flags': ['-fPIC'], 'directory': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C', 'external_dependencies': ['python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'language': 'c', 'libtype': 'shared', 'linker_language': 'c', 'source': 'python_wrapper.c'}, 'regex': {'directory': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_posix': {'directory': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c', 'libtype': 'header_only', 'source': 'regex_posix.h'}, 'regex_win32': {'directory': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/regex', 'external_dependencies': [], 'internal_dependencies': [], 'language': 'c++', 'libtype': 'object', 'source': 'regex_win32.cpp'}, 'ygg': {'compiler_flags': ['-fPIC'], 'external_dependencies': ['rapidjson', 'python', 'numpy'], 'include_dirs': ['/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/communication', '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/serialize', '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C', '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C'], 'internal_dependencies': ['regex', 'datatypes', 'python_wrapper'], 'language': 'c', 'linker_language': 'c++', 'source': '/opt/hostedtoolcache/Python/3.8.12/x64/lib/python3.8/site-packages/yggdrasil/languages/C/YggInterface.c'}}
inverse_type_map = None
is_typed = True
language = 'c'
language_ext = ['.c', '.h']
locked_buildfile = 'datatypes.o'
classmethod output2input(var, in_definition=True)[source]

Perform conversion necessary to turn an output variable into an corresponding input that can be used to format a function definition.

Parameters
  • var (dict) – Variable definition.

  • in_definition (bool, optional) – If True, the returned dictionary corresponds to an input variable in a function definition. If False, the returned value will correspond to an input to a function. Defaults to True.

Returns

Updated variable definition.

Return type

dict

outputs_in_inputs = True
classmethod parse_var_definition(io, value, **kwargs)[source]

Extract information about input/output variables from a string definition.

Parameters
  • io (str) – Description of variables contained in the provided string. Must be ‘inputs’ or ‘outputs’.

  • value (str) – String containing one or more variable definitions.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

List of information about the variables contained in

the provided string.

Return type

list

Raises
classmethod prepare_output_variables(vars_list, in_definition=False, in_inputs=False, for_yggdrasil=False)[source]

Concatenate a set of output variables such that it can be passed as a single string to the function_call parameter.

Parameters
  • vars_list (list) – List of variable names to concatenate as output from a function call.

  • in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying output variables in a function definition. Defaults to False.

  • in_inputs (bool, optional) – If True, the output variables should be formated to be included as input variables. Defaults to False.

  • for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns

Concatentated variables list.

Return type

str

classmethod prepare_variables(vars_list, in_definition=False, for_yggdrasil=False)[source]

Concatenate a set of input variables such that it can be passed as a single string to the function_call parameter.

Parameters
  • vars_list (list) – List of variable dictionaries containing info (e.g. names) that should be used to prepare a string representing input/output to/from a function call.

  • in_definition (bool, optional) – If True, the returned sequence will be of the format required for specifying variables in a function definition. Defaults to False.

  • for_yggdrasil (bool, optional) – If True, the variables will be prepared in the formated expected by calls to yggdarsil send/recv methods. Defaults to False.

Returns

Concatentated variables list.

Return type

str

classmethod requires_length_var(var)[source]

Determine if a variable requires a separate length variable.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if a length variable is required, False otherwise.

Return type

bool

classmethod requires_shape_var(var)[source]

Determine if a variable requires a separate shape variable.

Parameters

var (dict) – Dictionary of variable properties.

Returns

True if a shape variable is required, False otherwise.

Return type

bool

classmethod set_env_class(**kwargs)[source]

Set environment variables that are instance independent.

Parameters

**kwargs – Additional keyword arguments are passed to the parent class’s method and update_ld_library_path.

Returns

Environment variables for the model process.

Return type

dict

supported_comm_options = {'ipc': {'platforms': ['MacOS', 'Linux']}, 'zmq': {'libraries': ['zmq', 'czmq']}}
supported_comms = ['ipc', 'zmq']
type_class_map = {}
type_map = {'1darray': '*', 'any': 'generic_t', 'array': 'json_array_t', 'boolean': 'bool', 'bytes': 'char*', 'class': 'python_class_t', 'comm': 'comm_t*', 'complex': 'complex_X', 'dtype': 'dtype_t*', 'flag': 'int', 'float': 'double', 'function': 'python_function_t', 'instance': 'python_instance_t', 'int': 'intX_t', 'integer': 'int', 'ndarray': '*', 'null': 'void*', 'obj': 'obj_t', 'object': 'json_object_t', 'ply': 'ply_t', 'schema': 'schema_t', 'string': 'string_t', 'uint': 'uintX_t', 'unicode': 'unicode_t'}
classmethod update_ld_library_path(env, paths_to_add=None, add_to_front=False, add_libpython_dir=False, toolname=None, env_var=None, **kwargs)[source]

Update provided dictionary of environment variables so that LD_LIBRARY_PATH includes the interface directory containing the interface libraries.

Parameters
  • env (dict) – Dictionary of enviroment variables to be updated.

  • paths_to_add (list, optional) – Paths that should be added. If not provided, defaults to [cls.get_language_dir()].

  • add_to_front (bool, optional) – If True, new paths are added to the front, rather than the end. Defaults to False.

  • add_libpython_dir (bool, optional) – If True, the directory containing the Python C library will be added. Defaults to False.

  • toolname (str, optional) – Name of compiler tool that should be used. Defaults to None and the default compiler for the language will be used.

  • env_var (str, optional) – Environment variable where the paths should be added. Defaults to None and is only set for linux (LD_LIBRARY_PATH) and windows (PATH).

  • **kwargs – Additional keyword arguments are ignored.

Returns

Updated dictionary of environment variables.

Return type

dict

classmethod update_python_path(env)[source]

Update provided dictionary of environment variables so that PYTHONPATH and PYTHONHOME are set as needed (primarily on windows).

Parameters

env (dict) – Dictionary of enviroment variables to be updated.

Returns

Updated dictionary of environment variables.

Return type

dict

classmethod write_assign_to_output(dst_var, src_var, outputs_in_inputs=False, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]

Write lines assigning a value to an output variable.

Parameters
  • dst_var (str, dict) – Name or information dictionary for variable being assigned to.

  • src_var (str, dict) – Name or information dictionary for value being assigned to dst_var.

  • outputs_in_inputs (bool, optional) – If True, outputs are passed as input parameters. In some languages, this means that a pointer or reference is passed (e.g. C) and so the assignment should be to the memory indicated rather than the variable. Defaults to False.

  • dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.

  • use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines achieving assignment.

Return type

list

classmethod write_declaration(var, **kwargs)[source]

Return the lines required to declare a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Addition keyword arguments are passed to the parent class’s method.

Returns

The lines declaring the variable.

Return type

list

classmethod write_free(var, **kwargs)[source]

Return the lines required to free a variable with a certain type.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being declared.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines freeing the variable.

Return type

list

classmethod write_function_def(function_name, dont_add_lengths=False, use_length_prefix=False, **kwargs)[source]

Write a function definition.

Parameters
  • function_name (str) – Name fo the function being defined.

  • dont_add_lengths (bool, optional) – If True, length variables are not added for arrays. Defaults to False.

  • use_length_prefix (bool, optional) – If True and length variables are added, they will be named using prefixes. Otherwise, suffixes will be used. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines completing the function call.

Return type

list

Raises

ValueError – If outputs_in_inputs is not True and more than one output variable is specified.

classmethod write_initialize_oiter(var, value=None, **kwargs)[source]

Get the lines necessary to initialize an array for iteration output.

Parameters
  • var (dict, str) – Name or information dictionary for the variable being initialized.

  • value (str, optional) – Value that should be assigned to the variable.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

The lines initializing the variable.

Return type

list

classmethod write_model_function_call(model_function, flag_var, inputs, outputs, **kwargs)[source]

Write lines necessary to call the model function.

Parameters
  • model_function (str) – Handle of the model function that should be called.

  • flag_var (str) – Name of variable that should be used as a flag.

  • inputs (list) – List of dictionaries describing inputs to the model.

  • outputs (list) – List of dictionaries describing outputs from the model.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a call to a model function in

this language.

Return type

list

classmethod write_model_recv(channel, recv_var, **kwargs)[source]

Write a model receive call include checking the return flag.

Parameters
  • channel (str) – Name of variable that the channel being received from was stored in.

  • recv_var (dict, list) – Information of one or more variables that receieved information should be stored in.

  • **kwargs – Additional keyword arguments are passed to the parent class’s method.

Returns

Lines required to carry out a receive call in this language.

Return type

list

classmethod write_print_output_var(var, in_inputs=False, **kwargs)[source]

Get the lines necessary to print an output variable in this language.

Parameters
  • var (dict) – Variable information.

  • in_inputs (bool, optional) – If True, the output variable is passed in as an input variable to be populated. Defaults to False.

  • **kwargs – Additional keyword arguments are passed to write_print_var.

Returns

Lines printing the specified variable.

Return type

list

class yggdrasil.drivers.CModelDriver.ClangCompiler(**kwargs)[source]

Bases: yggdrasil.drivers.CModelDriver.CCompilerBase

Interface class for clang compiler/linker.

combine_with_linker = True
compatible_toolsets = ['llvm']
default_archiver = 'libtool'
default_executable = 'clang'
default_flags = ['-g', '-Wall']
default_linker = 'clang'
flag_options = {'definitions': '-D%s', 'include_dirs': '-I%s', 'isysroot': {'key': '-isysroot', 'prepend': True}, 'mmacosx-version-min': '-mmacosx-version-min=%s', 'sysroot': '--sysroot'}
classmethod get_flags(*args, **kwargs)[source]

Get a list of compiler flags.

is_gnu = False
is_linker = False
linker_attributes = {'default_executable_env': 'LD', 'default_flags_env': 'LDFLAGS', 'search_path_envvar': ['LIBRARY_PATH', 'LD_LIBRARY_PATH'], 'search_path_flags': ['-Xlinker', '--verbose'], 'search_regex': ['SEARCH_DIR\\("=([^"]+)"\\);']}
platforms = ['MacOS', 'Linux', 'Windows']
toolname = 'clang'
toolset = 'llvm'
class yggdrasil.drivers.CModelDriver.ClangLinker(**kwargs)[source]

Bases: yggdrasil.drivers.CModelDriver.LDLinker

Interface class for clang linker (calls to ld).

aliases = []
static before_registration(cls)[source]

Operations that should be performed to modify class attributes prior to registration including things like platform dependent properties and checking environment variables for default settings.

compatible_toolsets = ['llvm']
default_executable = 'clang'
default_flags = []
flag_options = {'library_dirs': '-L%s', 'library_libs': {'allow_duplicate_values': True, 'key': '-l%s'}, 'library_libs_nonstd': '', 'library_rpath': '-rpath', 'linker-version': '-mlinker-version=%s'}
classmethod get_flags(*args, **kwargs)[source]

Get a list of linker flags.

is_gnu = False
languages = ['c']
library_ext = '.so'
platforms = ['MacOS', 'Linux', 'Windows']
search_path_flags = ['-Xlinker', '-v']
search_regex = ['\\t([^\\t\\n]+)\\n']
search_regex_begin = 'Library search paths:'