yggdrasil.communication.transforms package

Submodules

yggdrasil.communication.transforms.ArrayTransform module

class yggdrasil.communication.transforms.ArrayTransform.ArrayTransform(*args, **kwargs)

Bases: TransformBase

Class for consolidating values into an array.

Parameters:

field_names (list, optional) – Names of fields in the array.

classmethod check_array_items(items, order=None, items_as_columns=None)

Check that items are valid types for array columns.

Parameters:

• items (list) – Type definitions for elements.

• order (list, optional) – Order that properties should be compared in for object schemas. Defaults to None and will be set based on the order of the keys in the first element (non-deterministic for Python 2.7).

• items_as_columns (bool, optional) – If True, the items will be parsed under the assumption that each item contains the schema describing a column, possible as an array of elements. If None and the initial check fails when assuming items are rows, columns will be tried. Defaults to None.

Raises:

AssertionError – If the items are not valid.

classmethod check_element(items, subtype=False)

Check that all elements in set of elements (e.g. row or column) are consistent.

Parameters:

• items (list) – Set of element type definitions.

• subtype (bool, optional) – If True, subtype, precision, and title information are used in the comparison. Defaults to False. subtype should be True if checking column elements and False if checking row elements.

Raises:

AssertionError – If any elements are not consistent.

classmethod check_summary(a, aidx, b, bidx)

Determine if two summary structures are equivalent, printing differences in the error if they are not.

Parameters:

• a (dict) – Summary information for an element type defintion.

• aidx (int) – Index of element summarized by a that is used in the error message.

• b (dict) – Summary information for an element type defintion.

• bidx (int) – Index of element summarized by b that is used in the error message.

Raises:

AssertionError – If a and b are not equivalent.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_summary(x, subtype=False)

Get subset of information summarizing an array element that can be used for comparison with other elements in the same row/column.

Parameters:

• x (dict) – Type definition for an array element.

• subtype (bool, optional) – If True, the subtype, shape, and title information is included. Defaults to False.

Returns:

Information about the array element.

Return type:

dict

Raises:

AssertionError – If x is not a valid type defintion for an array element.

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

set_original_datatype(datatype)

Set datatype.

Parameters:

datatype (dict) – Datatype.

classmethod transform_array_items(items, order=None)

Transform elements in an array.

Parameters:

• items (list) – Set of type definitions for array rows or columns that should be transformed into type definitions for a set of array columns.

• order (list, optional) – Order in which properties should be added as columns for object type defintions. Defaults to None if not provided and the first object element will be used to get the order (non-deterministic on Python 2.7).

Returns:

Transformed array column type definitions.

Return type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

validate_datatype(datatype)

Assert that the provided datatype is valid for this transformation.

Parameters:

datatype (dict) – Datatype to validate.

Raises:

AssertionError – If the datatype is not valid.

yggdrasil.communication.transforms.DirectTransform module

class yggdrasil.communication.transforms.DirectTransform.DirectTransform(*args, **kwargs)

Bases: TransformBase

Base class for message transforms.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

yggdrasil.communication.transforms.FilterTransform module

class yggdrasil.communication.transforms.FilterTransform.FilterTransform(*args, **kwargs)

Bases: TransformBase

Class for applying a filter.

Parameters:

filter (FilterBase) – Filter used to exclude some messages based on their contents.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

yggdrasil.communication.transforms.FunctionTransform module

class yggdrasil.communication.transforms.FunctionTransform.FunctionTransform(*args, **kwargs)

Bases: TransformBase

Class for transforming messages based on a provided Python function.

Parameters:

function (func) – The handle for a callable Python object (e.g. function) that should be used to transform messages or a string of the form “<function file>:<function name>” identifying a function where “<function file>” is the module or Python file containing the function and “<function name>” is the name of the function. The function should take the message as input and return the transformed message.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

yggdrasil.communication.transforms.IterateTransform module

class yggdrasil.communication.transforms.IterateTransform.IterateTransform(*args, **kwargs)

Bases: TransformBase

Class for iterating over message elements.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

get_elements(datatype)

Get a list of elements in the datatype for iteration.

Parameters:

datatype (dict) – Datatype to get elements from.

Returns:

List of datatypes for the elements iterated over.

Return type:

list

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

validate_datatype(datatype)

Assert that the provided datatype is valid for this transformation.

Parameters:

datatype (dict) – Datatype to validate.

Raises:

AssertionError – If the datatype is not valid.

yggdrasil.communication.transforms.MapFieldsTransform module

class yggdrasil.communication.transforms.MapFieldsTransform.MapFieldsTransform(*args, **kwargs)

Bases: TransformBase

Class for mapping a subset of the original fields in an object to a different set of fields. Fields that don’t exist in the map are preserved unchanged.

Parameters:

map (dict) – A mapping from original field name to new field names.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

yggdrasil.communication.transforms.MapTransform module

class yggdrasil.communication.transforms.MapTransform.MapTransform(*args, **kwargs)

Bases: TransformBase

Class for transforming an object into a dictionary.

Parameters:

field_names (list, optional) – A list of field names that should be used for array object. If not provided, names will be generated according to ‘f0’, ‘f1’, ‘f2’, etc.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

yggdrasil.communication.transforms.PandasTransform module

class yggdrasil.communication.transforms.PandasTransform.PandasTransform(*args, **kwargs)

Bases: ArrayTransform

Class for consolidating values into a Pandas data frame.

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options()

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

yggdrasil.communication.transforms.SelectFieldsTransform module

class yggdrasil.communication.transforms.SelectFieldsTransform.SelectFieldsTransform(*args, **kwargs)

Bases: TransformBase

Class for selecting a subset of the original fields in an object.

Parameters:

• selected (list) – A list of fields that should be selected.

• original_order (list, optional) – The original order of fields that should be used for selecting from lists/tuples.

• single_as_scalar (bool, optional) – If True and only a single field is selected, the transformed messages will be scalars rather than arrays with single elements. Defaults to False.

property as_single

True if there is a single element to return.

Type:

bool

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

set_original_datatype(datatype)

Set datatype.

Parameters:

datatype (dict) – Datatype.

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

validate_datatype(datatype)

Assert that the provided datatype is valid for this transformation.

Parameters:

datatype (dict) – Datatype to validate.

Raises:

AssertionError – If the datatype is not valid.

yggdrasil.communication.transforms.SelectScalarTransform module

class yggdrasil.communication.transforms.SelectScalarTransform.SelectScalarTransform(*args, **kwargs)

Bases: SelectFieldsTransform

Class for selecting a single element from an array or dict and returning it as a scalar.

Parameters:

index (int, string, optional) – Array index or dictionary key to select. Defaults to selecting the first element in an array or the first key alphabetically.

evaluate_transform(x, **kwargs)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• **kwargs – Additional keyword arguments are passed to the parent class.

Returns:

The transformed message.

Return type:

object

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

property selected

Selected fields for use by base class.

Type:

list

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

yggdrasil.communication.transforms.StatementTransform module

class yggdrasil.communication.transforms.StatementTransform.StatementTransform(*args, **kwargs)

Bases: TransformBase

Class for transforming messages based on a provided statement using Python syntax.

Parameters:

statement (str) – Python statement in terms of the message as represented by the string “%x%” that should evaluate to the transformed message. The statement should only use a limited set of builtins and the math library (See yggdrasil.tools.safe_eval). If more complex relationships are required, use the FunctionTransform class.

statement

Python statement that will be evaluated to transform messages.

Type:

str

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

True if the message will pass through the transform, False otherwise.

Return type:

bool

classmethod get_testing_options(**kwargs)

Get testing options for the transform class.

Returns:

Multiple dictionaries of keywords and messages before/after

pairs that will result from the transform created by the provided keywords.

Return type:

list

yggdrasil.communication.transforms.TransformBase module

class yggdrasil.communication.transforms.TransformBase.TransformBase(*args, **kwargs)

Bases: ComponentBase

Base class for message transforms.

Parameters:

original_datatype (dict, optional) – Datatype associated with expected messages. Defaults to None.

__call__(x, no_init=False, **kwargs)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_init (bool, optional) – If True, the datatype is not initialized if it is not already set. Defaults to False.

• **kwargs – Additional keyword arguments are passed to call_transform.

Returns:

The transformed message.

Return type:

object

call_transform(x, no_init=False, **kwargs)

Call transform, setting datatypes during the process.

Parameters:

• x (object) – Message object to transform.

• no_init (bool, optional) – If True, the datatype is not initialized if it is not already set. Defaults to False.

• **kwargs – Additional keyword arguments are passed to evaluate_transform.

Returns:

The transformed message.

Return type:

object

evaluate_transform(x, no_copy=False)

Call transform on the provided message.

Parameters:

• x (object) – Message object to transform.

• no_copy (bool, optional) – If True, the transformation occurs in place. Otherwise a copy is created and transformed. Defaults to False.

Returns:

The transformed message.

Return type:

object

set_original_datatype(datatype)

Set datatype.

Parameters:

datatype (dict) – Datatype.

set_original_datatype_from_data(data)

Set datatype from data.

Parameters:

data (object) – Data object.

set_transformed_datatype(datatype)

Set datatype.

Parameters:

datatype (dict) – Datatype.

set_transformed_datatype_from_data(data)

Set datatype from data.

Parameters:

data (object) – Data object.

transform_datatype(datatype)

Determine the datatype that will result from applying the transform to the supplied datatype.

Parameters:

datatype (dict) – Datatype to transform.

Returns:

Transformed datatype.

Return type:

dict

property transformed_datatype

The transformed datatype.

Type:

dict

validate_datatype(datatype)

Assert that the provided datatype is valid for this transformation.

Parameters:

datatype (dict) – Datatype to validate.

Raises:

AssertionError – If the datatype is not valid.

Module contents