Python Code¶

Subpackages¶

Submodules¶

cis_interface.backwards module¶

This module allows for backward compatibility.

cis_interface.backwards.as_str(s_in, recurse=False, convert_types=None, allow_pass=False)[source]¶

Convert from bytes/unicode/str to a str object.

Parameters:	s_in (str, bytes, unicode) – Object to convert to a str version. recurse (bool, optional) – If True, objects that are lists, tuples, or dicts will be recursed into, converting string elements into unicode. convert_types (list, optional) – Python types that should be converted to unicode. Defaults to the string types for the current Python version ([str, unicode, bytearray] in Python 2, [bytes, str, bytearray] otherwise). allow_pass (bool, optional) – If True and the provided object is not converted, no error will be raised. Defaults to False and objects that cannot be converted will raise an error.
Returns:	Str version of input.
Return type:	str
Raises:	TypeError – If supplied type cannot be converted to str.

cis_interface.backwards.as_bytes(s_in, recurse=False, convert_types=None, allow_pass=False)[source]¶

Convert from bytes/unicode/str to a bytes object.

Parameters:	s_in (str, bytes, unicode) – Object to convert to a bytes version. recurse (bool, optional) – If True, objects that are lists, tuples, or dicts will be recursed into, converting string elements into unicode. convert_types (list, optional) – Python types that should be converted to unicode. Defaults to the string types for the current Python version ([str, unicode, bytearray] in Python 2, [bytes, str, bytearray] otherwise). allow_pass (bool, optional) – If True and the provided object is not converted, no error will be raised. Defaults to False and objects that cannot be converted will raise an error.
Returns:	Bytes version of input (str in Python 2, bytes in Python 3).
Return type:	bytes/str
Raises:	TypeError – If supplied type cannot be converted to bytes.

cis_interface.backwards.as_unicode(s_in, recurse=False, convert_types=None, allow_pass=False)[source]¶

Convert from bytes/unicode/str to unicode.

Parameters:	s_in (bytes, unicode, str) – Object to be converted into unicode. recurse (bool, optional) – If True, objects that are lists, tuples, or dicts will be recursed into, converting string elements into unicode. convert_types (list, optional) – Python types that should be converted to unicode. Defaults to the string types for the current Python version ([str, unicode, bytearray] in Python 2, [bytes, str, bytearray] otherwise). allow_pass (bool, optional) – If True and the provided object is not converted, no error will be raised. Defaults to False and objects that cannot be converted will raise an error.
Returns:	Unicode version of input (unicode in Python 2, str in Python 3).
Return type:	unicode/str
Raises:	TypeError – If supplied type cannot be converted to unicode and allow_pass is False.

cis_interface.command_line module¶

cis_interface.command_line.cc_flags()[source]¶

Get the compiler flags necessary for including the interface library in a C or C++ program.

Returns:	The necessary compiler flags and preprocessor definitions.
Return type:	list

cis_interface.command_line.ciscc()[source]¶

Compile C/C++ program.

cis_interface.command_line.cisrun()[source]¶

Start a run.

cis_interface.command_line.cistime_comm()[source]¶

Plot timing statistics comparing the different communication mechanisms.

cis_interface.command_line.cistime_lang()[source]¶

Plot timing statistics comparing the different languages.

cis_interface.command_line.cistime_os()[source]¶

Plot timing statistics comparing the different operating systems.

cis_interface.command_line.cistime_py()[source]¶

Plot timing statistics comparing the different versions of Python.

cis_interface.command_line.ld_flags()[source]¶

Get the linker flags necessary for calling functions/classes from the interface library in a C or C++ program.

Returns:	The necessary library linking flags.
Return type:	list

cis_interface.command_line.rebuild_c_api()[source]¶

Rebuild the C/C++ API.

cis_interface.command_line.regen_schema()[source]¶

Regenerate the cis_interface schema.

cis_interface.command_line.update_config()[source]¶

Update the user config file for cis_interface.

cis_interface.command_line.validate_yaml()[source]¶

Validate a set of or or more YAMLs defining an integration.

cis_interface.config module¶

This module imports the configuration for cis_interface.

class cis_interface.config.CisConfigParser(defaults=None, dict_type=<class 'collections.OrderedDict'>, allow_no_value=False, *, delimiters=('=', ':'), comment_prefixes=('#', ';'), inline_comment_prefixes=None, strict=True, empty_lines_in_values=True, default_section='DEFAULT', interpolation=<object object>, converters=<object object>)[source]¶

Bases: configparser.ConfigParser

Config parser that returns None if option not provided on get.

get(section, option, default=None, **kwargs)[source]¶

Return None if the section/option does not exist.

Parameters:	section (str) – Name of section. option (str) – Name of option in section. default (obj, optional) – Value that should be returned if the section and/or option are not found or are an empty string. Defaults to None. **kwargs – Additional keyword arguments are passed to the parent class’s get.
Returns:	String entry if the section & option exist, otherwise default.
Return type:	obj

cis_interface.config.cfg_environment(env=None, cfg=None)[source]¶

Set environment variables based on config options.

Parameters:	env (dict, optional) – Dictionary of environment variables that should be updated. Defaults to os.environ. cfg (cis_interface.config.CisConfigParser, optional) – Config parser with options that should be used to update the environment. Defaults to cis_interface.config.cis_cfg.

cis_interface.config.cfg_logging(cfg=None)[source]¶

Set logging levels from config options.

Parameters:	cfg (cis_interface.config.CisConfigParser, optional) – Config parser with options that should be used to update the environment. Defaults to cis_interface.config.cis_cfg.

cis_interface.config.find_all(name, path)[source]¶

Find all instances of a file with a given name within the directory tree starting at a given path.

Parameters:	name (str) – Name of the file to be found (with the extension). path (str, None) – Directory where search should start. If set to None on Windows, the current directory and PATH variable are searched.
Returns:	All instances of the specified file.
Return type:	list

cis_interface.config.locate_file(fname)[source]¶

Locate a file on PATH.

Parameters:	fname (str) – Name of the file that should be located.
Returns:	Full path to the located file if it was located, False otherwise.
Return type:	bool, str

cis_interface.config.update_config(config_file, config_base=None)[source]¶

Update config options for the current platform.

Parameters:	config_file (str) – Full path to the config file that should be created and/or updated. config_base (str, optional) – Full path to existing config file that should be used as a base for building the new one if it dosn’t already exist. Defaults to ‘defaults.cfg’ if not provided.

cis_interface.config.update_config_c(config)[source]¶

Update config options specific to C/C++.

Parameters:	config (CisConfigParser) – Config class that options should be set for.
Returns:	Section, option, description tuples for options that could not be set.
Return type:	list

cis_interface.config.update_config_matlab(config)[source]¶

Update config options specific to matlab.

Parameters:	config (CisConfigParser) – Config class that options should be set for.
Returns:	Section, option, description tuples for options that could not be set.
Return type:	list

cis_interface.config.update_config_windows(config)[source]¶

Update config options specific to windows.

Parameters:	config (CisConfigParser) – Config class that options should be set for.
Returns:	Section, option, description tuples for options that could not be set.
Return type:	list

cis_interface.platform module¶

This module handle platform compatibility issues.

cis_interface.runner module¶

This module provides tools for running models using cis_interface.

class cis_interface.runner.CisRunner(modelYmls, namespace, host=None, rank=0, cis_debug_level=None, rmq_debug_level=None, cis_debug_prefix=None)[source]¶

Bases: cis_interface.tools.CisClass

This class handles the orchestration of starting the model and IO drivers, monitoring their progress, and cleaning up on exit.

Parameters:

modelYmls (list) – List of paths to yaml files specifying the models that should be run. namespace (str) – Name that should be used to uniquely identify any RMQ exchange. host (str, optional) – Name of the host that the models will be launched from. Defaults to None. rank (int, optional) – Rank of this set of models if run in parallel. Defaults to 0. cis_debug_level (str, optional) – Level for CiS debug messages. Defaults to environment variable ‘CIS_DEBUG’. rmq_debug_level (str, optional) – Level for RabbitMQ debug messages. Defaults to environment variable ‘RMQ_DEBUG’. cis_debug_prefix (str, optional) – Prefix for CiS debug messages. Defaults to namespace.

namespace¶

Name that should be used to uniquely identify any RMQ exchange.

Type:	str

host¶

Name of the host that the models will be launched from.

Type:	str

rank¶

Rank of this set of models if run in parallel.

Type:	int

modeldrivers¶

Model drivers associated with this run.

Type:	dict

inputdrivers¶

Input drivers associated with this run.

Type:	dict

outputdrivers¶

Output drivers associated with this run.

Type:	dict

serverdrivers¶

The addresses associated with different server drivers.

Type:	dict

interrupt_time¶

Time of last interrupt signal.

Type:	float

error_flag¶

True if one or more models raises an error.

Type:	bool

..todo:: namespace, host, and rank do not seem strictly necessary.

all_drivers¶

For all drivers.

Type:	iterator

cleanup()[source]¶

Perform cleanup operations for all drivers.

closeChannels(force_stop=False)[source]¶

Stop IO drivers and join the threads.

Parameters:	force_stop (bool, optional) – If True, the terminate method is used to stop the drivers. Otherwise, the stop method is used. The stop method will try to exit gracefully while terminate will exit as quickly as possible. Defaults to False.

createDriver(yml)[source]¶

Create a driver instance from the yaml information.

Parameters:	yml (yaml) – Yaml object containing driver information.
Returns:	An instance of the specified driver.
Return type:	object

createInputDriver(yml)[source]¶

Create an input driver instance from the yaml information.

Parameters:	yml (yaml) – Yaml object containing driver information.
Returns:	An instance of the specified driver.
Return type:	object

createModelDriver(yml)[source]¶

Create a model driver instance from the yaml information.

Parameters:	yml (yaml) – Yaml object containing driver information.
Returns:	An instance of the specified driver.
Return type:	object

createOutputDriver(yml)[source]¶

Create an output driver instance from the yaml information.

Parameters:	yml (yaml) – Yaml object containing driver information.
Returns:	An instance of the specified driver.
Return type:	object

do_client_exits(model)[source]¶

Perform exits for IO drivers associated with a client model.

Parameters:	model (dict) – Dictionary of model parameters including any associated IO drivers.

do_model_exits(model)[source]¶

Perform exits for IO drivers associated with a model.

Parameters:	model (dict) – Dictionary of model parameters including any associated IO drivers.

io_drivers(model=None)[source]¶

Return the input and output drivers for one or all models.

Parameters:	model (str, optional) – Name of a model that I/O drivers should be returned for. Defaults to None and all I/O drivers are returned.
Returns:	Access to list of I/O drivers.
Return type:	iterator

loadDrivers()[source]¶

Load all of the necessary drivers, doing the IO drivers first and adding IO driver environmental variables back tot he models.

pprint(*args)[source]¶

Print with color.

printStatus()[source]¶

Print the status of all drivers, starting with the IO drivers.

reset_signal_handler()[source]¶

Reset signal handlers to old ones.

run(signal_handler=None, timer=None, t0=None)[source]¶

Run all of the models and wait for them to exit.

Parameters:	signal_handler (function, optional) – Function that should be used as a signal handler. Defaults to None and is set by set_signal_handler. timer (function, optional) – Function that should be called to get intermediate timing statistics. Defaults to time.time if not provided. t0 (float, optional) – Zero point for timing statistics. Is set using the provided timer if not provided.
Returns:	Intermediate times from the run.
Return type:	dict

set_signal_handler(signal_handler=None)[source]¶

Set the signal handler.

Parameters:	signal_handler (function, optional) – Function that should handle received SIGINT and SIGTERM signals. Defaults to self.signal_handler.

signal_handler(sig, frame)[source]¶

Terminate all drivers on interrrupt.

startDrivers()[source]¶

Start drivers, starting with the IO drivers.

terminate()[source]¶

Immediately stop all drivers, beginning with IO drivers.

waitModels()[source]¶

Wait for all model drivers to finish. When a model finishes, join the thread and perform exits for associated IO drivers.

cis_interface.runner.get_runner(models, **kwargs)[source]¶

Get runner for a set of models, getting run information from the environment.

Parameters:	models (list) – List of yaml files containing information on the models that should be run. **kwargs – Additonal keyword arguments are passed to CisRunner.
Returns:	Runner for the provided models.
Return type:	CisRunner
Raises:	Exception – If config option ‘namespace’ in ‘rmq’ section not set.

cis_interface.scanf module¶

Small scanf implementation.

Python has powerful regular expressions but sometimes they are totally overkill when you just want to parse a simple-formatted string. C programmers use the scanf-function for these tasks (see link below).

This implementation of scanf translates the simple scanf-format into regular expressions. Unlike C you can be sure that there are no buffer overflows possible.

For more information see

• http://www.python.org/doc/current/lib/node49.html
• http://en.wikipedia.org/wiki/Scanf

Original code from:

http://code.activestate.com/recipes/502213-simple-scanf-implementation/

Modified original to make the %f more robust, as well as added %* modifier to skip fields.

Version: 1.3.3

Releases:

1.0

2010-10-11

• Initial release

1.1

2010-10-13

• Changed regex from ‘match’ (only matches at beginning of line) to ‘search’ (matches anywhere in line)
• Bugfix - ignore cast for skipped fields

1.2

2013-05-30

• Added ‘collapseWhitespace’ flag (defaults to True) to take the search string and replace all whitespace with regex string to match repeated whitespace. This enables better matching in log files where the data has been formatted for easier reading. These cases have variable amounts of whitespace between the columns, depending on the number of characters in the data itself.

1.3

2016-01-18

• Add ‘extractdata’ function.

1.3.1

2016-06-23

• Release to PyPi, now including README.md

Updates by Meagan Lang:

2018-04-12

• Added ability to parse components of field specifiers and cope with whitespace padding.

cis_interface.scanf.scanf(format, s=None, collapseWhitespace=True)[source]¶

scanf supports the following formats:

%c One character %5c 5 characters %d, %i int value %7d, %7i int value with length 7 %f float value %o octal value %X, %x hex value %s string terminated by whitespace

Examples: >>> scanf(“%s - %d errors, %d warnings”, “/usr/sbin/sendmail - 0 errors, 4 warnings”) (‘/usr/sbin/sendmail’, 0, 4) >>> scanf(“%o %x %d”, “0123 0x123 123”) (66, 291, 123)

If the parameter s is a file-like object, s.readline is called. If s is not specified, stdin is assumed.

The function returns a tuple of found values or None if the format does not match.

cis_interface.scanf.scanf_compile(format, collapseWhitespace=True)[source]¶

Translate the format into a regular expression

For example: >>> format_re, casts = _scanf_compile(‘%s - %d errors, %d warnings’) >>> print format_re.pattern (S+) - ([+-]?d+) errors, ([+-]?d+) warnings

Translated formats are cached for faster reuse

cis_interface.schema module¶

class cis_interface.schema.ComponentSchema(schema_type, schema_registry=None, schema_subtypes=None)[source]¶

Bases: object

Schema information for one component.

Parameters:	schema_type (str) – The name of the component. schema_registry (SchemaRegistry, optional) – Registry of schemas that this schema is dependent on. **kwargs – Additional keyword arguments are entries in the component schema.

append(comp_cls, subtype=None)[source]¶

Append component class to the schema.

Parameters:	comp_cls (class) – Component class that should be added. subtype (str, tuple, optional) – Key used to identify the subtype of the component type. Defaults to subtype_attr if one was provided, otherwise the subtype will not be logged.

class2subtype¶

Mapping from class to list of subtypes.

Type:	dict

classes¶

All available classes for this schema.

Type:	list

classmethod from_registry(schema_type, schema_classes, **kwargs)[source]¶

Construct a ComponentSchema from a registry entry.

Parameters:	schema_type (str) – Name of component type to build. schema_classes (list) – List of classes for the component type. **kwargs – Additional keyword arguments are passed to the class __init__ method.
Returns:	Schema with information from classes.
Return type:	ComponentSchema

classmethod from_schema(schema, schema_registry=None)[source]¶

Construct a ComponentSchema from a schema.

Parameters:	schema (dict) – Schema.
Returns:	Schema with information from schema.
Return type:	ComponentSchema

full_schema¶

Schema for evaluating YAML input file that fully specifies the properties for each component.

Type:	dict

get_subtype_properties(subtype)[source]¶

Get the valid properties for a specific subtype.

Parameters:	subtype (str) – Name of the subtype to get keys for.
Returns:	Valid properties for the specified subtype.
Return type:	list

properties¶

Valid properties for this component.

Type:	list

schema¶

Schema for this component.

Type:	dict

subtype2class¶

Mapping from subtype to class.

Type:	dict

subtypes¶

All subtypes for this schema type.

Type:	list

class cis_interface.schema.SchemaRegistry(registry=None, required=None)[source]¶

Bases: object

Registry of schema’s for different integration components.

Parameters:	registry (dict, optional) – Dictionary of registered components. Defaults to None and the registry will be empty. required (list, optional) – Components that are required. Defaults to [‘comm’, ‘file’, ‘model’, ‘connection’]. Ignored if registry is None.
Raises:	ValueError – If registry is provided and one of the required components is missing.

add(k, v)[source]¶

Add a new component schema to the registry.

classmethod from_file(fname)[source]¶

Create a SchemaRegistry from a file.

Parameters:	fname (str) – Full path to the file the schema should be loaded from.

full_schema¶

Schema for evaluating YAML input file that fully specifies the properties for each component.

Type:	dict

get(k)[source]¶

Return a component schema from the registry.

get_component_keys(comp_name)[source]¶

Get the properties associated with a certain component.

Parameters:	comp_name (str) – Name of the component to return keys for.
Returns:	All of the valid properties for the specified component.
Return type:	list

get_component_schema(comp_name)[source]¶

Get the schema for a certain component.

Parameters:	comp_name (str) – Name of the component to get the schema for.
Returns:	Schema for the specified component.
Return type:	dict

is_valid_component(comp_name, obj)[source]¶

Determine if an object is a valid represenation of a component.

Parameters:	comp_name (str) – Name of the component to validate against. obj (object) – Object to validate.
Returns:	True if the object is valid, False otherwise.
Return type:	bool

keys()[source]¶

load(fname)[source]¶

Load schema from a file.

Parameters:	fname (str) – Full path to the file the schema should be loaded from.

normalize(obj, backwards_compat=False, **kwargs)[source]¶

Normalize an object against this schema.

Parameters:	obj (object) – Object to normalize. **kwargs – Additional keyword arguments are passed to normalize_instance.
Returns:	Normalized object.
Return type:	object

classmethod register_normalizer(path)[source]¶

Register a normalizer that will be applied to elements in the instance at the specified path.

Parameters:	path (tuple) – Location in schema where normalizer will be applied.
Returns:	Decorator for registering the normalizer function.
Return type:	function

save(fname)[source]¶

Save the schema to a file.

Parameters:	fname (str) – Full path to the file the schema should be saved to. schema (dict) – cis_interface YAML options.

schema¶

Schema for evaluating YAML input file.

Type:	dict

validate(obj, **kwargs)[source]¶

Validate an object against this schema.

Parameters:	obj (object) – Object to valdiate. **kwargs – Additional keyword arguments are passed to validate_instance.

validate_component(comp_name, obj)[source]¶

Validate an object against a specific component.

Parameters:	comp_name (str) – Name of the component to validate against. obj (object) – Object to validate.

cis_interface.schema.cdriver2commtype(driver)[source]¶

Convert a connection driver to a file type.

Parameters:	driver (str) – The name of the connection driver.
Returns:	The corresponding file type for the driver.
Return type:	str

cis_interface.schema.cdriver2filetype(driver)[source]¶

Convert a connection driver to a file type.

Parameters:	driver (str) – The name of the connection driver.
Returns:	The corresponding file type for the driver.
Return type:	str

cis_interface.schema.clear_schema()[source]¶

Clear global schema.

cis_interface.schema.create_schema()[source]¶

Create a new schema from the registry.

cis_interface.schema.get_schema(fname=None)[source]¶

Return the cis_interface schema for YAML options.

Parameters:	fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
Returns:	cis_interface YAML options.
Return type:	dict

cis_interface.schema.inherit_schema(orig, new_values=None, remove_keys=None, **kwargs)[source]¶

Create an inherited schema, adding new value to accepted ones for dependencies.

Parameters:	orig (dict) – Schema that will be inherited. new_values (dict, optional) – Dictionary of new values to add. Defaults to None and is ignored. remove_keys (list, optional) – Keys that should be removed form orig before adding the new keys. Defaults to empty list. **kwargs – Additional keyword arguments will be added to the schema with dependency on the provided key/value pair.
Returns:	New schema.
Return type:	dict

cis_interface.schema.init_registry()[source]¶

Initialize the registries and schema.

cis_interface.schema.init_schema(fname=None)[source]¶

Initialize global schema.

cis_interface.schema.load_schema(fname=None)[source]¶

Return the cis_interface schema for YAML options.

Parameters:	fname (str, optional) – Full path to the file that the schema should be loaded from. If the file dosn’t exist, it is created. Defaults to _schema_fname.
Returns:	cis_interface YAML options.
Return type:	dict

cis_interface.schema.migrate_keys(from_dict, to_dict, key_list)[source]¶

Migrate keys from one component to another that are not in a list of predefined keys.

Parameters:	from_dict (dict) – Component dictionary to migrate keys from. to_dict (dict) – List of component dictionaries to migrate keys to. key_list (list) – List of allowable keys for the original component. All keys in the original that are not in this list will be moved.

cis_interface.schema.ordered_dump(data, stream=None, Dumper=<class 'yaml.dumper.Dumper'>, **kwds)[source]¶

cis_interface.schema.ordered_load(stream, Loader=<class 'yaml.loader.Loader'>, object_pairs_hook=<class 'collections.OrderedDict'>)[source]¶

cis_interface.schema.register_component(component_class)[source]¶

Decorator for registering a class as a yaml component.

cis_interface.schema.rwmeth2filetype(rw_meth)[source]¶

Get the alternate properties that corresponding to the old read_meth/write_meth keywords.

Parameters:	rw_meth (str) – Read/write method name.
Returns:	Property values equivalent to provided read/write method.
Return type:	dict

cis_interface.schema.standardize(instance, keys, is_singular=False, suffixes=None, altkeys=None)[source]¶

Standardize a component such that each key contains a list of dictionaries.

Parameters:

instance (dict) – Component to standardize. keys (list) – Keys to standardize in the instance. is_singular (bool, optional) – If False, the keys are assumed to be plural and singular alternatives are also checked. If True, the keys are assumed to be singular and plural alternatives are also checked. Defaults to False. suffixes (list, optional) – Suffixes to add to the keys to get a set of alternate keys that should also be checked. Defaults to None as is ignored. altkeys (list, optional) – List of lists of alternate keys. Defaults to None.

cis_interface.timing module¶

class cis_interface.timing.TimedRun(lang_src, lang_dst, test_name='timed_pipe', filename=None, comm_type=None, platform=None, python_ver=None, max_errors=5, matlab_running=False, dont_use_perf=False, **kwargs)[source]¶

Bases: cis_interface.tests.CisTestBase, cis_interface.tools.CisClass

Class to time sending messages from one language to another.

Parameters:

lang_src (str) – Language that messages should be sent from. lang_dst (str) – Language that messages should be sent to. test_name (str, optional) – Name of the example. Defaults to ‘timed_pipe’. filename (str, optional) – Full path to the file where timing statistics will be saved. This can be a perf json or a Python pickle of run data. Defaults to ‘scalings_{test_name}_{comm_type}.json’ if dont_use_perf is False, otherwise the extension is ‘.dat’. comm_type (str, optional) – Name of communication class that should be used for tests. Defaults to the current default comm class. platform (str, optional) – Platform that the test should be run on. If the data doesn’t already exist and this doesn’t match the current platform, an error will be raised. Defaults to the current platform. python_ver (str, optional) – Version of Python that the test should be run with. If the data doesn’t already exist and this doesn’t match the current version of python, an error will be raised. Defaults to the current version of python. max_errors (int, optional) – Maximum number of errors that should be retried. Defaults to 5. matlab_running (bool, optional) – If True, the test will assert that there is an existing Matlab engine before starting, otherwise the test will assert that there is not an existing Matlab engine. Defaults to False. dont_use_perf (bool, optional) – If True, the timings will be run without using the perf package. Defaults to False.

lang_src¶

Language that messages should be sent from.

Type:	str

lang_dst¶

Language that messages should be sent to.

Type:	str

platform¶

Platform that the test is being run on.

Type:	str

python_ver¶

Version of Python that the test should be run with.

Type:	str

filename¶

Full path to the file where timing statistics will be saved. This can be a perf json or a Python pickle of run data.

Type:	str

comm_type¶

Name of communication class that should be used for tests.

Type:	str

max_errors¶

Maximum number of errors that should be retried.

Type:	int

matlab_running¶

True if there was a Matlab engine running when the test was created. False otherwise.

Type:	bool

dont_use_perf¶

If True, the timings will be run without using the perf package.

Type:	bool

after_run(run_uuid, result)[source]¶

Actions that should be performed after a run.

Parameters:	nmsg (int) – Number of messages that were sent. msg_size (int) – Size of each message that were sent. result (float) – Time required (in seconds) to execute the program.

base_msg_count¶

Message count to use for tests varrying message size.

Type:	int

base_msg_size¶

Message size to use for tests varrying message count.

Type:	int

before_run(nmsg, msg_size)[source]¶

Actions that should be performed before a run.

Parameters:	nmsg (int) – Number of messages that should be sent. msg_size (int) – Size of each message that should be sent.
Returns:	Unique identifier for the run.
Return type:	str

can_run(raise_error=False)[source]¶

Determine if the test can be run from the current platform and python version.

Parameters:	raise_error (bool, optional) – If True, an error will be raised if the test cannot be completed from the current platform. Defaults to False.
Returns:	True if the test can be run, False otherwise.
Return type:	bool

check_output(fout, nmsg, msg_size)[source]¶

Assert that the output file contains the expected result.

Parameters:	fout (str) – The file that should be checked. nmsg (int) – The number of messages that will be sent. msg_sizze (int) – The size of the the messages that will be sent.

classmethod class_plot(lang_src='python', lang_dst='python', **kwargs)[source]¶

Create the class for a given combo of languages and comm types, then call plot_scaling_joint.

Parameters:	lang_src (str, optional) – Language that messages should be sent from. Defaults to ‘python’. lang_dst (str, optional) – Language that messages should be sent to. Defaults to ‘python’. **kwargs – Additional keywords are passed to either the class constructor or plot_scaling_joint as appropriate.
Returns:	Pair of axes containing the plotted scalings and the fit.
Return type:	tuple(matplotlib.Axes, matplotlib.Axes)

cleanup_output(fout)[source]¶

Cleanup the output file.

Parameters:	fout (str) – The file to be cleaned up.

data¶

Timing statistics data.

Type:	dict or perf.BenchmarkSuite

default_msg_count¶

Default message count for scaling tests.

Type:	list

default_msg_size¶

Default message sizes for scaling tests. This will vary depending on the comm type so that the maximum size is not more than 10x larger than the maximum message size.

Type:	list

entry_name(nmsg, msg_size)[source]¶

Get a unique identifier for a run.

Parameters:	nmsg (int) – Number of messages that should be sent. msg_size (int) – Size of each message that should be sent.

fit_scaling_count(msg_size=None, counts=None, **kwargs)[source]¶

Do a linear fit to the scaling of execution time with message count.

Parameters:	msg_size (int, optional) – Size of each message that should be sent. Defaults to self.base_msg_size. counts (list, optional) – List of counts to test. Defaults to self.default_msg_count if not provided. **kwargs – Additional keyword arguments are passed to scaling_count.
Returns:	The slope and intercept of the linear fit.
Return type:	tuple

fit_scaling_size(msg_count=None, sizes=None, **kwargs)[source]¶

Do a linear fit to the scaling of execution time with message count.

Parameters:	msg_count (int, optional) – Number of messages that should be sent for each size. Defaults to self.base_msg_count. sizes (list, optional) – List of sizes to test. Defaults to self.default_msg_size if not provided. **kwargs – Additional keyword arguments are passed to scaling_size.
Returns:	The slope and intercept of the linear fit.
Return type:	tuple

get_entry(name)[source]¶

Get values for an entry.

Parameters:	name (str) – Name of the entry to return.

get_new_uuid()[source]¶

Get a new unique ID.

Returns:	Unique identifier.
Return type:	str

get_yaml_dst(lang)[source]¶

Get the yaml entry for the destination model.

Parameters:	lang (str) – Language for the destination model.

get_yaml_src(lang)[source]¶

Get the yaml entry for the source model.

Parameters:	lang (str) – Language for the source model.

has_entry(name)[source]¶

Check to see if there is an entry with the provided name.

Parameters:	name (str) – Name of the entry to check for.

load(as_json=False)[source]¶

Load scalings data from a perf BenchmarkSuite json file or a Python pickle.

Parameters:	as_json (bool, optional) – If True and self.dont_use_perf is False, the perf BenchmarkSuite data will be loaded as a dictionary from the json. Defaults to False.
Returns:	Loaded scalings data. None is returned if the file does not exit.
Return type:	perf.BenchmarkSuite or dict

make_yamlfile(path)[source]¶

Create a YAML file for running the test.

Parameters:	path (str) – Full path to file where the YAML should be saved.

max_msg_size¶

Largest size of message that can be sent without being split.

Type:	int

output_content(nmsg, msg_size)[source]¶

Get the result that should be output to file during the run.

Parameters:	nmsg – The number of messages that will be sent. msg_sizze – The size of the the messages that will be sent.
Returns:	The contents expected in the file.
Return type:	str

output_file_format¶

Full path to the output file created by the run.

Type:	str

perfscript¶

Format string for creating a perf script.

Type:	str

plot_scaling(msg_size, msg_count, axs=None, label=None, xscale=None, yscale='linear', plot_kws={}, time_method='average', per_message=False, **kwargs)[source]¶

Plot scaling of run time with a variable.

Parameters:	msg_size (int, list, np.ndarray) – List of message sizes to use as x variable, or message size to use when plotting dependent on message count. msg_count (int, list, np.ndarray) – List of message counts to use as x variable, or message count to use when plotting dependent on message size. axs (matplotlib.Axes, optional) – Axes object that line should be added to. If not provided, one is created. label (str, optional) – Label that should be used for the line. Defaults to None. xscale (str, optional) – ‘log’ or ‘linear’ to indicate what scale the x axis should use. Defaults to ‘linear’. yscale (str, optional) – ‘log’ or ‘linear’ to indicate what scale the y axis should use. Defaults to ‘linear’. plot_kws (dict, optional) – Ploting keywords that should be passed. Defaults to {}. time_method (str, optional) – Timing method that should be used. Valid values include ‘bestof’ and ‘average’. Defaults to ‘average’. per_message (bool, optional) – If True, the time per message is returned rather than the total time. Defaults to False. **kwargs – Additional keyword arguments are passed to scaling_size or scaling_count.
Returns:	Axes containing the plotted scaling.
Return type:	matplotlib.Axes

plot_scaling_joint(msg_size0=None, msg_count0=None, msg_size=None, msg_count=None, axs=None, **kwargs)[source]¶

Plot scaling of run time with both count and size, side by side. Anywhere data is exchanged as a tuple for each plot, the plot of scaling with count is first and the scaling with size is second.

Parameters:	msg_size0 (int, optional) – Size of messages to use for count scaling. Defaults to self.base_msg_size. msg_count0 (int, optional) – Number of messages to use for size scaling. Defaults to self.base_msg_count. msg_size (list, np.ndarray, optional) – List of message sizes to use as x variable on the size scaling plot. Defaults to self.default_msg_size. msg_count (list, np.ndarray, optional)) – List of message counts to use as x variable on the count scaling plot. Defaults to self.default_msg_count. axs (tuple, optional) – Pair of axes objects that lines should be added to. If not provided, they are created. **kwargs – Additional keyword arguments are passed to plot_scaling.
Returns:	Pair of axes containing the plotted scalings and the fit.
Return type:	tuple(matplotlib.Axes, matplotlib.Axes)

reload()[source]¶

Reload scalings data and store it in the data attribute.

remove_entry(name)[source]¶

Remove all runs associated with an entry.

Parameters:	name (str) – Name of the entry to be removed.

run(run_uuid, timer=<built-in function time>, t0=None)[source]¶

Run test sending a set of messages between the designated models.

Parameters:	run_uuid (str) – Unique ID for the run. timer (function, optional) – Function that should be called to get intermediate timing statistics. Defaults to time.time if not provided. t0 (float, optional) – Zero point for timing statistics. Is set using the provided timer if not provided.
Returns:	Intermediate times from the run.
Return type:	dict

save(data, overwrite=False)[source]¶

Save scalings data to a new perf BenchmarkSuite json file or a Python pickle. If the file exists and overwrite is not set, an error will be raised. No file is written if data is None.

Parameters:	data (perf.BenchmarkSuite or dict) – Data to be saved. overwrite (bool, optional) – If True, any existing file will be overwritten. Defaults to False.
Raises:	RuntimeError – If the file already exists and overwrite is False.

scaling_count(msg_size, counts=None, min_count=1, max_count=100, nsamples=10, scaling='linear', per_message=False, **kwargs)[source]¶

Get scaling of run time with message count.

Parameters:	msg_size (int) – Size of each message that should be sent. counts (list, optional) – List of counts to test. Defaults to None and a list is created based on the other keyword arguments. min_count (int, optional) – Minimum message count that should be timed. Defaults to 1. This is ignored if ‘counts’ is provided. max_count (int, optional) – Maximum message count that should be timed. Defaults to 100. This is ignored if ‘counts’ is provided. nsamples (int, optional) – Number of samples that should be done between ‘min_count’ and ‘max_count’. Defaults to 10. This is ignored if ‘counts’ is provided. scaling (str, optional) – Scaling for sampling of message counts between ‘min_count’ and ‘max_count’. Defaults to ‘linear’. This is ignored if ‘counts’ is provided. per_message (bool, optional) – If True, the time per message is returned rather than the total time. Defaults to False. **kwargs – Additional keyword arguments are passed to time_run.
Returns:	Lists of counts timed, minimum execution time, average execution times, and standard deviations.
Return type:	tuple

scaling_size(nmsg, sizes=None, min_size=1, max_size=10000000.0, nsamples=10, scaling='log', per_message=False, **kwargs)[source]¶

Get scaling of run time with message size.

Parameters:	nmsg (int) – Number of messages that should be sent. sizes (list, optional) – List of sizes to test. Defaults to None and a list is created based on the other keyword arguments. min_size (int, optional) – Minimum message size that should be timed. Defaults to 1. This is ignored if ‘sizes’ is provided. max_size (int, optional) – Maximum message size that should be timed. Defaults to 1e7. This is ignored if ‘sizes’ is provided. nsamples (int, optional) – Number of samples that should be done between ‘min_size’ and ‘max_size’. Defaults to 10. This is ignored if ‘sizes’ is provided. scaling (str, optional) – Scaling for sampling of message sizes between ‘min_size’ and ‘max_size’. Defaults to ‘linear’. This is ignored if ‘sizes’ is provided. per_message (bool, optional) – If True, the time per message is returned rather than the total time. Defaults to False. **kwargs – Additional keyword arguments are passed to time_run.
Returns:	Lists of sizes timed, minimum execution times, average execution times, and standard deviations.
Return type:	tuple

source_dst¶

Source file for language messages will be sent to.

Type:	str

source_src¶

Source file for language messages will be sent from.

Type:	str

startup_time¶

Time required to set up communications and start models.

Type:	float

tempdir¶

Temporary directory.

Type:	str

time_per_byte¶

time_per_message¶

Time required to send a single message of 1000 bytes.

Type:	float

time_run(nmsg, msg_size, nrep=10, overwrite=False)[source]¶

Time sending a set of messages between the designated models.

Parameters:	nmsg (int) – Number of messages that should be sent. msg_size (int) – Size of each message that should be sent. nrep (int, optional) – Number of times the test should be repeated to get an average execution time and standard deviation. Defaults to 10. overwrite (bool, optional) – If True, any existing entry for this run will be overwritten. Defaults to False.
Returns:	Best of, average and standard deviation in the time (in seconds) required to execute the program.
Return type:	tuple

time_run_mine(nmsg, msg_size, nrep)[source]¶

Time sending a set of messages between the designated models without using the perf package.

Parameters:	nmsg (int) – Number of messages that should be sent. msg_size (int) – Size of each message that should be sent. nrep (int) – Number of times the test should be repeated. These reps will be appended to the existing reps for this entry.

time_run_perf(nmsg, msg_size, nrep)[source]¶

Time sending a set of messages between the designated models using the perf package.

Parameters:	nmsg (int) – Number of messages that should be sent. msg_size (int) – Size of each message that should be sent. nrep (int) – Number of times the test should be repeated. These reps will be appended to the existing reps for this entry.

yamlfile_format¶

Format string for creating a yaml file.

Type:	str

cis_interface.timing.get_source(lang, direction, test_name='timed_pipe')[source]¶

Get the path to the source file.

Parameters:	lang (str) – Language that should be returned. direction (str) – ‘src’ or ‘dst’. test_name (str, optional) – Name of the example. Defaults to ‘timed_pipe’.
Returns:	Full path to the source file.
Return type:	str

cis_interface.timing.perf_func(loops, timer, nmsg, msg_size, max_errors)[source]¶

Function to do perf loops over function.

Parameters:	loops (int) – Number of loops to perform. timer (TimedRun) – Class with information about the run and methods required for setup/teardown. nmsg (int) – Number of messages that should be sent in the test. msg_size (int) – Size of messages that should be sent in the test. max_errors (int) – Maximum number of errors that should be retried.
Returns:	Time (in seconds) required to perform the test the required number of times.
Return type:	float

cis_interface.timing.perfjson_to_pandas(json_file)[source]¶

Convert perf benchmarks json file to a Pandas data frame.

Parameters:	json_file (str) – Full path to the JSON benchmarks file that should be added.
Returns:	Data frame version of perf benchmarks.
Return type:	pandas.DataFrame

cis_interface.timing.plot_scalings(compare='comm_type', compare_values=None, plotfile=None, test_name='timed_pipe', cleanup_plot=False, use_paper_values=False, **kwargs)[source]¶

Plot comparison of scaling for chosen variable.

Parameters:	compare (str, optional) – Name of variable that should be compared. Valid values are ‘language’, ‘comm_type’, ‘platform’, ‘python_ver’. Defaults to ‘comm_type’. compare_values (list, optional) – Values that should be plotted. If not provided, the values will be determined based on the current platform. plotfile (str, optional) – Full path to the file where the plot will be saved. If not provided, one is created based on the test parameters in the current working directory. test_name (str, optional) – Name of the test that should be used. Defaults to ‘timed_pipe’. cleanup_plot (bool, optional) – If True, the create plotfile will be removed before the function returns. This is generally only useful for testing. Defaults to False. use_paper_values (bool, optional) – If True, use the values from the paper. Defaults to False. **kwargs – Additional keyword arguments are passed to plot_scaling_joint.
Returns:	Path where the figure was saved.
Return type:	str

cis_interface.timing.write_perf_script(script_file, nmsg, msg_size, lang_src, lang_dst, comm_type, nrep=10, max_errors=5, matlab_running=False)[source]¶

Write a script to run perf.

Parameters:

script_file (str) – Full path to the file where the script should be saved. nmsg (int) – The number of messages that should be sent during the run. msg_size (int) – The size (in bytes) of the test messages that should be sent during the run. lang_src (str) – The language that the source program should be in. lang_dst (str) – The language that the destination program should be in. comm_type (str) – The type of communication channel that should be used for the test. nrep (int, optional) – The number of times the test run should be repeated. Defaults to 3. max_errors (int, optional) – Maximum number of errors that should be retried. Defaults to 5. matlab_running (bool, optional) – If True, the test will assert that there is an existing Matlab engine before starting, otherwise the test will assert that there is not an existing Matlab engine. Defaults to False.

cis_interface.tools module¶

This modules offers various tools.

class cis_interface.tools.CisClass(name=None, uuid=None, working_dir=None, timeout=60.0, sleeptime=0.01, **kwargs)[source]¶

Bases: logging.LoggerAdapter

Base class for CiS classes.

Parameters:

name (str) – Class name. uuid (str, optional) – Unique ID for this instance. Defaults to None and is assigned. working_dir (str, optional) – Working directory. If not provided, the current working directory is used. timeout (float, optional) – Maximum time (in seconds) that should be spent waiting on a process. Defaults to 60. sleeptime (float, optional) – Time that class should sleep for when sleep is called. Defaults to 0.01. **kwargs – Additional keyword arguments are assigned to the extra_kwargs dictionary.

name¶

Class name.

Type:	str

uuid¶

Unique ID for this instance.

Type:	str

sleeptime¶

Time that class should sleep for when sleep called.

Type:	float

longsleep¶

Time that the class will sleep for when waiting for longer tasks to complete (10x longer than sleeptime).

Type:	float

timeout¶

Maximum time that should be spent waiting on a process.

Type:	float

working_dir¶

Working directory.

Type:	str

errors¶

List of errors.

Type:	list

extra_kwargs¶

Keyword arguments that were not parsed.

Type:	dict

sched_out¶

Output from the last scheduled task with output.

Type:	obj

logger¶

Logger object for this object.

Type:	logging.Logger

suppress_special_debug¶

If True, special_debug log messages are suppressed.

Type:	bool

as_str(obj)[source]¶

Return str version of object if it is not already a string.

Parameters:	obj (object) – Object that should be turned into a string.
Returns:	String version of provided object.
Return type:	str

check_timeout(key=None, key_level=0)[source]¶

Check timeout for the calling function/method.

Parameters:

key (str, optional) – Key for timeout that should be checked. Defaults to None and is set by the calling class and function/method (See timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0.

Raises:

KeyError – If there is not a timeout registered for the specified key.

cis_class¶

Name of the class.

Type:	str

debug_log()[source]¶

Turn on debugging.

display(msg='', *args, **kwargs)[source]¶

Print a message, no log.

dummy_log(*args, **kwargs)[source]¶

Dummy log function that dosn’t do anything.

error¶

Log an error level message.

exception¶

Log an exception level message.

get_timeout_key(key_level=0, key_suffix=None)[source]¶

Return a key for a given level in the stack, relative to the function calling get_timeout_key.

Parameters:	key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that is 2 steps higher in the stack. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.
Returns:	Key identifying calling object and method.
Return type:	str

name

Name of the class object.

Type:	str

periodic_debug(key, period=10)[source]¶

Log that should occur periodically rather than with every call.

Parameters:	key (str) – Key that should be used to identify the debug message. period (int, optional) – Period (in number of messages) that messages should be logged at. Defaults to 10.
Returns:	Logging method to be used.
Return type:	method

pprint(obj, block_indent=0, indent_str=' ', **kwargs)[source]¶

Use pprint to represent an object as a string.

Parameters:	obj (object) – Python object to represent. block_indent (int, optional) – Number of indents that should be placed in front of the entire block. Defaults to 0. indent_str (str, optional) – String that should be used to indent. Defaults to 4 spaces. **kwargs – Additional keyword arguments are passed to pprint.pformat.
Returns:	String representation of object using pprint.
Return type:	str

printStatus()[source]¶

Print the class status.

print_encoded(msg, *args, **kwargs)[source]¶

Print bytes to stdout, encoding if possible.

Parameters:	msg (str, bytes) – Message to print. *args – Additional arguments are passed to print. **kwargs – Additional keyword arguments are passed to print.

process(msg, kwargs)[source]¶

Process logging message.

reset_log()[source]¶

Resetting logging to prior value.

sched_task(t, func, args=None, kwargs=None, store_output=False)[source]¶

Schedule a task that will be executed after a certain time has elapsed.

Parameters:

t (float) – Number of seconds that should be waited before task is executed. func (object) – Function that should be executed. args (list, optional) – Arguments for the provided function. Defaults to []. kwargs (dict, optional) – Keyword arguments for the provided function. Defaults to {}. store_output (bool, optional) – If True, the output from the scheduled task is stored in self.sched_out. Otherwise, it is not stored. Defaults to False.

sleep(t=None)[source]¶

Have the class sleep for some period of time.

Parameters:	t (float, optional) – Time that class should sleep for. If not provided, the attribute ‘sleeptime’ is used.

special_debug¶

Log debug level message contingent of supression flag.

start_timeout(t=None, key=None, key_level=0, key_suffix=None)[source]¶

Start a timeout for the calling function/method.

Parameters:

t (float, optional) – Maximum time that the calling function should wait before timeing out. If not provided, the attribute ‘timeout’ is used. key (str, optional) – Key that should be associated with the timeout that is created. Defaults to None and is set by the calling class and function/method (See get_timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored.

Raises:

KeyError – If the key already exists.

stop_timeout(key=None, key_level=0, key_suffix=None, quiet=False)[source]¶

Stop a timeout for the calling function method.

Parameters:

key (str, optional) – Key for timeout that should be stopped. Defaults to None and is set by the calling class and function/method (See timeout_key). key_level (int, optional) – Positive integer indicating the level of the calling class and function/method that should be used to key the timeout. 0 is the class and function/method that called start_timeout. Higher values use classes and function/methods further up in the stack. Defaults to 0. key_suffix (str, optional) – String that should be appended to the end of the generated key. Defaults to None and is ignored. quiet (bool, optional) – If True, error message on timeout exceeded will be debug log. Defaults to False.

Raises:

KeyError – If there is not a timeout registered for the specified key.

timeout_key¶

Key identifying calling object and method.

Type:	str

verbose_debug(*args, **kwargs)[source]¶

Log a verbose debug level message.

class cis_interface.tools.CisPopen(cmd_args, forward_signals=True, **kwargs)[source]¶

Bases: subprocess.Popen

Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe. In addition this class overrides Popen.kill() to allow processes to be killed with CTRL_BREAK_EVENT on windows.

Parameters:	args (list, str) – Shell command or list of arguments that should be run. forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True. **kwargs – Additional keywords arguments are passed to Popen.

kill(*args, **kwargs)[source]¶

On windows using CTRL_BREAK_EVENT to kill the process.

class cis_interface.tools.CisThread(name=None, target=None, args=(), kwargs=None, daemon=False, group=None, **cis_kwargs)[source]¶

Bases: threading.Thread, cis_interface.tools.CisClass

Thread for CiS that tracks when the thread is started and joined.

lock¶

Lock for accessing the sockets from multiple threads.

Type:	threading.RLock

start_event¶

Event indicating that the thread was started.

Type:	threading.Event

terminate_event¶

Event indicating that the thread should be terminated. The target must exit when this is set.

Type:	threading.Event

atexit()[source]¶

Actions performed when python exits.

before_start()[source]¶

Actions to perform on the main thread before starting the thread.

cleanup()[source]¶

Actions to perform to clean up the thread after it has stopped.

main_terminated¶

True if the main thread has terminated.

Type:	bool

run(*args, **kwargs)[source]¶

Continue running until terminate event set.

set_started_flag()[source]¶

Set the started flag for the thread to True.

set_terminated_flag()[source]¶

Set the terminated flag for the thread to True.

start(*args, **kwargs)[source]¶

Start thread and print info.

terminate(no_wait=False)[source]¶

Set the terminate event and wait for the thread to stop.

Parameters:	no_wait (bool, optional) – If True, terminate will not block until the thread stops. Defaults to False and blocks.
Raises:	AssertionError – If no_wait is False and the thread has not stopped after the timeout.

unset_started_flag()[source]¶

Set the started flag for the thread to False.

unset_terminated_flag()[source]¶

Set the terminated flag for the thread to False.

wait(timeout=None, key=None)[source]¶

Wait until thread finish to return using sleeps rather than blocking.

Parameters:	timeout (float, optional) – Maximum time that should be waited for the driver to finish. Defaults to None and is infinite. key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.

was_started¶

True if the thread was started. False otherwise.

Type:	bool

was_terminated¶

True if the thread was terminated. False otherwise.

Type:	bool

class cis_interface.tools.CisThreadLoop(*args, **kwargs)[source]¶

Bases: cis_interface.tools.CisThread

Thread that will run a loop until the terminate event is called.

after_loop()[source]¶

Actions performed after the loop.

before_loop()[source]¶

Actions performed before the loop.

on_main_terminated(dont_break=False)[source]¶

Actions performed when 1st main terminated.

Parameters:	dont_break (bool, optional) – If True, the break flag won’t be set. Defaults to False.

run(*args, **kwargs)[source]¶

Continue running until terminate event set.

run_loop(*args, **kwargs)[source]¶

Actions performed on each loop iteration.

set_break_flag()[source]¶

Set the break flag for the thread to True.

set_loop_flag()[source]¶

Set the loop flag for the thread to True.

terminate(*args, **kwargs)[source]¶

Also set break flag.

unset_break_flag()[source]¶

Set the break flag for the thread to False.

unset_loop_flag()[source]¶

Set the loop flag for the thread to False.

wait_for_loop(timeout=None, key=None)[source]¶

Wait until thread enters loop to return using sleeps rather than blocking.

Parameters:	timeout (float, optional) – Maximum time that should be waited for the thread to enter loop. Defaults to None and is infinite. key (str, optional) – Key that should be used to register the timeout. Defaults to None and is set based on the stack trace.

was_break¶

True if the break flag was set.

Type:	bool

was_loop¶

True if the thread was loop. False otherwise.

Type:	bool

class cis_interface.tools.TimeOut(max_time, key=None)[source]¶

Bases: object

Class for checking if a period of time has been elapsed.

Parameters:	max_time (float, bbol) – Maximum period of time that should elapse before ‘is_out’ returns True. If False, ‘is_out’ will never return True. Providing 0 indicates that ‘is_out’ should immediately return True. key (str, optional) – Key that was used to register the timeout. Defaults to None.

max_time¶

Maximum period of time that should elapsed before ‘is_out’ returns True.

Type:	float

start_time¶

Result of time.time() at start.

Type:	float

key¶

Key that was used to register the timeout.

Type:	str

elapsed¶

Total time that has elapsed since the start.

Type:	float

is_out¶

True if there is not any time remaining. False otherwise.

Type:	bool

cis_interface.tools.check_locks()[source]¶

Check for locks in lock registry that are locked.

cis_interface.tools.check_sockets()[source]¶

Check registered sockets.

cis_interface.tools.check_threads()[source]¶

Check for threads that are still running.

cis_interface.tools.cis_atexit()[source]¶

Things to do at exit.

cis_interface.tools.eval_kwarg(x)[source]¶

If x is a string, eval it. Otherwise just return it.

Parameters:	x (str, obj) – String to be evaluated as an object or an object.
Returns:	Result of evaluated string or the input object.
Return type:	obj

cis_interface.tools.get_CIS_MSG_MAX(comm_type=None)[source]¶

Get the maximum message size for a given comm type.

Parameters:	comm_type (str, optional) – The name of the communication type that the maximum message size should be returned for. Defaults to result of get_default_comm() if not provided.
Returns:	Maximum message size (in bytes).
Return type:	int

cis_interface.tools.get_default_comm()[source]¶

Get the default comm that should be used for message passing.

cis_interface.tools.get_installed_comm(language=None)[source]¶

Get a list of the communication channel types that are supported by cis_interface on the current machine. This checks the operating system, supporting libraries, and broker credentials. The order indicates the prefered order of use.

Parameters:	language (str, optional) – Specific programming language that installed comms should be located for. Defaults to None and all languages supported on the current platform will be checked.
Returns:	The names of the the communication channel types supported on the current machine.
Return type:	list

cis_interface.tools.get_installed_lang()[source]¶

Get a list of the languages that are supported by cis_interface on the current machine. This checks for the necessary interpreters, licenses, and/or compilers.

Returns:	The name of languages supported on the current machine.
Return type:	list

cis_interface.tools.get_supported_comm()[source]¶

Get a list of the communication mechanisms supported by cis_interface.

Returns:	The names of communication mechanisms supported by cis_interface.
Return type:	list

cis_interface.tools.get_supported_lang()[source]¶

Get a list of the model programming languages that are supported by cis_interface.

Returns:	The names of programming languages supported by cis_interface.
Return type:	list

cis_interface.tools.is_comm_installed(comm, language=None)[source]¶

Check to see if cis_interface can use a communication mechanism on the current machine.

Parameters:	comm (str) – Communication mechanism to check. language (str, optional) – Specific programming language that communication mechanism should be check for. Defaults to None and all supported languages will be checked.
Returns:	True if the communication mechanism can be used on the current machine, False otherwise.
Return type:	bool

cis_interface.tools.is_lang_installed(lang)[source]¶

Check to see if cis_interface can run models written in a programming language on the current machine.

Parameters:	lang (str) – Programming language to check.
Returns:	True if models in the provided language can be run on the current machine, False otherwise.
Return type:	bool

cis_interface.tools.kill(pid, signum)[source]¶

Kill process by mapping signal number.

Parameters:	pid (int) – Process ID. signum (int) – Signal that should be sent.

cis_interface.tools.locate_path(fname, basedir='/')[source]¶

Find the full path to a file using where on Windows.

cis_interface.tools.popen_nobuffer(*args, **kwargs)[source]¶

Uses Popen to open a process without a buffer. If not already set, the keyword arguments ‘bufsize’, ‘stdout’, and ‘stderr’ are set to 0, subprocess.PIPE, and subprocess.STDOUT respectively. This sets the output stream to unbuffered and directs both stdout and stderr to the stdout pipe.

Parameters:	args (list, str) – Shell command or list of arguments that should be run. forward_signals (bool, optional) – If True, flags will be set such that signals received by the spawning process will be forwarded to the child process. If False, the signals will not be forwarded. Defaults to True. **kwargs – Additional keywords arguments are passed to Popen.
Returns:	Process that was started.
Return type:	CisPopen

cis_interface.tools.print_encoded(msg, *args, **kwargs)[source]¶

Print bytes to stdout, encoding if possible.

Parameters:	msg (str, bytes) – Message to print. *args – Additional arguments are passed to print. **kwargs – Additional keyword arguments are passed to print.

cis_interface.tools.sleep(interval)[source]¶

Sleep for a specified number of seconds.

Parameters:	interval (float) – Time in seconds that process should sleep.

cis_interface.units module¶

cis_interface.units.add_units(arr, unit_str, dtype=None)[source]¶

Add units to an array or scalar.

Parameters:	arr (np.ndarray, float, int) – Scalar or array of data to add units to. unit_str (str) – Unit string. dtype (np.dtype, optional) – Numpy data type that should be maintained for array/qunatity with units. If not provided, this is determined from the array.
Returns:	Array with units.
Return type:	unyt.unyt_array

cis_interface.units.are_compatible(units1, units2)[source]¶

Check if two units are compatible.

Parameters:	units1 (str) – First units string. units2 (str) – Second units string.
Returns:	True if the units are compatible, False otherwise.
Return type:	bool

cis_interface.units.as_unit(ustr)[source]¶

Get unit object for the string.

Args:

Returns:

Raises:	ValueError – If the string is not a recognized unit.

cis_interface.units.convert_to(arr, new_units)[source]¶

Convert qunatity with units to new units. Objects without units will be returned with the new units.

Parameters:	arr (np.ndarray, float, int, unyt.unyt_array) – Quantity with or without units. new_units (str) – New units that should be applied.
Returns:	Array with new units.
Return type:	unyt.unyt_array

cis_interface.units.get_data(obj)[source]¶

Get the array/scalar assocaited with the object.

Parameters:	obj (object) – Object to get data for.
Returns:	Numpy array representation of the underlying data.
Return type:	np.ndarray

cis_interface.units.get_units(obj)[source]¶

Get the string representation of the units.

Parameters:	obj (object) – Object to get units for.
Returns:	Units, empty if input object has none.
Return type:	str

cis_interface.units.has_units(obj)[source]¶

Determine if a Python object has associated units.

Parameters:	obj (object) – Object to be tested for units.
Returns:	True if the object has units, False otherwise.
Return type:	bool

cis_interface.units.is_null_unit(ustr)[source]¶

Determines if a string is a null unit.

Parameters:	ustr (str) – String to test.
Returns:	True if the string is ‘’ or ‘n/a’, False otherwise.
Return type:	bool

cis_interface.units.is_unit(ustr)[source]¶

Determine if a string is a valid unit.

Parameters:	ustr (str) – String representation to test.
Returns:	True if the string is a valid unit. False otherwise.
Return type:	bool

cis_interface.yamlfile module¶

cis_interface.yamlfile.link_model_io(existing)[source]¶

Link I/O drivers back to the models they communicate with.

Parameters:	existing (dict) – Dictionary of existing components.
Returns:	Dictionary with I/O drivers added to models.
Return type:	dict

cis_interface.yamlfile.load_yaml(fname)[source]¶

Parse a yaml file defining a run.

Parameters:	fname (str) – Path to the yaml file.
Returns:	Contents of yaml file.
Return type:	dict

cis_interface.yamlfile.parse_component(yml, ctype, existing=None)[source]¶

Parse a yaml entry for a component, adding it to the list of existing components.

Parameters:	yml (dict) – YAML dictionary for a component. ctype (str) – Component type. This can be ‘input’, ‘output’, ‘model’, or ‘connection’. existing (dict, optional) – Dictionary of existing components. Defaults to empty dict.
Raises:	TypeError – If yml is not a dictionary. ValueError – If dtype is not ‘input’, ‘output’, ‘model’, or ‘connection’. ValueError – If the component already exists.
Returns:	All components identified.
Return type:	dict

cis_interface.yamlfile.parse_connection(yml, existing)[source]¶

Parse a yaml entry for a connection between I/O channels.

Parameters:	yml (dict) – YAML dictionary for a connection. existing (dict) – Dictionary of existing components.
Raises:	RuntimeError – If the ‘inputs’ entry is not a model output or file. RuntimeError – If neither the ‘inputs’ or ‘outputs’ entries correspond to model I/O channels.
Returns:	Updated log of all entries.
Return type:	dict

cis_interface.yamlfile.parse_model(yml, existing)[source]¶

Parse a yaml entry for a model.

Parameters:	yml (dict) – YAML dictionary for a model. existing (dict) – Dictionary of existing components.
Returns:	Updated log of all entries.
Return type:	dict

cis_interface.yamlfile.parse_yaml(files)[source]¶

Parse list of yaml files.

Parameters:	files (str, list) – Either the path to a single yaml file or a list of yaml files.
Raises:	ValueError – If the yml dictionary is missing a required keyword or has an invalid value. RuntimeError – If one of the I/O channels is not initialized with driver information.
Returns:	Dictionary of information parsed from the yamls.
Return type:	dict

cis_interface.yamlfile.prep_yaml(files)[source]¶

Prepare yaml to be parsed by Cerberus using schema including covering backwards compatible options.

Parameters:	files (str, list) – Either the path to a single yaml file or a list of yaml files.
Returns:	YAML ready to be parsed using schema.
Return type:	dict

Module contents¶

This package provides a framework for integrating models across languages such that they can be run simultaneously, passing input back and forth.