Command Line Interface (CLI)

yggdrasil Command

usage: yggdrasil [-h] [--version]
                 {run,info,validate,compile,compile-deps,clean,install,config,metaschema,schema,model-form-schema,dev-update,test,timing,gha,integration-service-manager}
                 ...

Named Arguments

--version

show program’s version number and exit

subcommands

subcommand

Possible choices: run, info, validate, compile, compile-deps, clean, install, config, metaschema, schema, model-form-schema, dev-update, test, timing, gha, integration-service-manager

Sub-commands:

run

Run an integration.

yggdrasil run [-h] [--with-mpi WITH_MPI] [--mpi-tag-start MPI_TAG_START]
              [--validate] [--loglevel LOGLEVEL] [--rmq-loglevel RMQ_LOGLEVEL]
              [--client-loglevel CLIENT_LOGLEVEL] [--validate-components]
              [--validate-messages {False,True,First}] [--namespace NAMESPACE]
              [--host HOST] [--vhost VHOST] [--user USER]
              [--password PASSWORD] [--cluster CLUSTER]
              [--default-comm DEFAULT_COMM] [--production-run] [--debug]
              yamlfile [yamlfile ...]
Positional Arguments
yamlfile

One or more yaml specification files.

Named Arguments
--with-mpi, --mpi-nproc

Number of MPI processes to run on.

Default: 1

--mpi-tag-start

Tag that MPI communications should start at.

Default: 0

--validate

Validate the run via model validation commands on completion.

Default: False

--loglevel

Logging level for yggdrasil operations.

--rmq-loglevel, --rmqloglevel

Logging level for RabbitMQ operations.

--client-loglevel, --clientloglevel

Logging level for yggdrasil operations on model processes.

--validate-components, --validatecomponents

Validate components on creation using their JSON schema (Decreases performance).

Default: False

--validate-messages, --validatemessages

Possible choices: False, True, First

Which messages should be validated during communication. ‘True’: all messages (decreases performance), ‘False’: no messages, or ‘First’: only the first message a comm sends/receives.

--namespace

RabbitMQ namespace.

--host

RabbitMQ host address.

--vhost

RabbitMQ virtual host address.

--user

RabbitMQ username.

--password

RabbitMQ password.

--cluster

Cluster that should be used.

--default-comm, --defaultcomm

Comm type that should be used by default.

--production-run

Turn off safe guards in order to improve performance. This is equivalent to ‘–validate-components –validate-messages=True’

Default: False

--debug

Turn on debugging utilties including increased logging and validation. This is equivalent to ‘–loglevel=DEBUG –client-loglevel=DEBUG –validate-components=False –validate-messages=False’

Default: False

info

Display information about the current yggdrasil installation.

yggdrasil info [-h] [--no-languages] [--no-comms] [--verbose]
               {compiler,linker,archiver} ...
Named Arguments
--no-languages

Don’t print information about individual languages.

Default: False

--no-comms

Don’t print information about individual comms.

Default: False

--verbose, -v

Increase the verbosity of the printed information.

Default: False

tool

Compilation tool types to get info about.

tool

Possible choices: compiler, linker, archiver

Sub-commands:
compiler

Get information about a compiler.

yggdrasil info compiler [-h] [--toolname TOOLNAME] [--flags] [--fullpath]
                        {c,c++,cpp,cxx,fortran}
Positional Arguments
language

Possible choices: c, c++, cpp, cxx, fortran

Language to get tool information for.

Named Arguments
--toolname

Name of tool to get information for. If not provided, information for the default tool will be returned.

--flags

Display the flags that yggdrasil will pass to the tool when it is called.

Default: False

--fullpath

Get the full path to the tool exectuable.

Default: False

linker

Get information about a linker.

yggdrasil info linker [-h] [--toolname TOOLNAME] [--flags] [--fullpath]
                      [--library]
                      {c,c++,cpp,cxx,fortran}
Positional Arguments
language

Possible choices: c, c++, cpp, cxx, fortran

Language to get tool information for.

Named Arguments
--toolname

Name of tool to get information for. If not provided, information for the default tool will be returned.

--flags

Display the flags that yggdrasil will pass to the tool when it is called.

Default: False

--fullpath

Get the full path to the tool exectuable.

Default: False

--library

Get flags for linking a library.

Default: False

archiver

Get information about a archiver.

yggdrasil info archiver [-h] [--toolname TOOLNAME] [--flags] [--fullpath]
                        {c,c++,cpp,cxx,fortran}
Positional Arguments
language

Possible choices: c, c++, cpp, cxx, fortran

Language to get tool information for.

Named Arguments
--toolname

Name of tool to get information for. If not provided, information for the default tool will be returned.

--flags

Display the flags that yggdrasil will pass to the tool when it is called.

Default: False

--fullpath

Get the full path to the tool exectuable.

Default: False

validate

Validate a set of YAML specification files for an integration.

yggdrasil validate [-h] [--model-only] [--model-submission]
                   yamlfile [yamlfile ...]
Positional Arguments
yamlfile

One or more YAML specification files.

Named Arguments
--model-only

Validate a YAML containing an isolated model without ensuring that it is part of a complete integration.

Default: False

--model-submission

Validate a YAML against the requirements for submissions to the yggdrasil model repository.

Default: False

compile

Compile a program from source files for use in an yggdrasil integration.

yggdrasil compile [-h] [--language {None,c,c++,cpp,cxx,fortran,R,r}]
                  [--toolname TOOLNAME] [--flags [FLAGS [FLAGS ...]]]
                  [--use-ccache] [--Rpkg-language RPKG_LANGUAGE]
                  source [source ...]
Positional Arguments
source

One or more source files or the directory containing an R package.

Named Arguments
--language

Possible choices: None, c, c++, cpp, cxx, fortran, R, r

Language of the source code. If not provided, the language will be determined from the source extension.

--toolname

Name of compilation tool that should be used

--flags

Additional flags that should be added to the compilation command

--use-ccache

Run compilation with ccache.

Default: False

--Rpkg-language

Language that R package is written in (only used if the specified language is R).

compile-deps

Compile yggdrasil dependency libraries. Existing libraries are first deleted.

yggdrasil compile-deps [-h] [--toolname TOOLNAME] [language [language ...]]
Positional Arguments
language

One or more languages to compile dependencies for.

Default: [‘all’]

Named Arguments
--toolname

Name of compilation tool that should be used

clean

Remove dependency libraries compiled by yggdrasil.

yggdrasil clean [-h] [language [language ...]]
Positional Arguments
language

One or more languages to clean up dependencies for.

Default: [‘all’]

install

Complete yggdrasil installation for one or more languages.

yggdrasil install [-h]

config

Update the user config file.

yggdrasil config [-h] [--languages LANGUAGES_FLAG [LANGUAGES_FLAG ...]]
                 [--show-file] [--remove-file] [--overwrite]
                 [--disable-languages {c,c++,cpp,cxx,fortran,R,matlab,python,r,cmake,make,lpy,osr,sbml,dummy,executable,mpi,timesync} [{c,c++,cpp,cxx,fortran,R,matlab,python,r,cmake,make,lpy,osr,sbml,dummy,executable,mpi,timesync} ...]]
                 [--enable-languages {c,c++,cpp,cxx,fortran,R,matlab,python,r,cmake,make,lpy,osr,sbml,dummy,executable,mpi,timesync} [{c,c++,cpp,cxx,fortran,R,matlab,python,r,cmake,make,lpy,osr,sbml,dummy,executable,mpi,timesync} ...]]
                 [--quiet] [--allow-multiple-omp] [--dont-allow-multiple-omp]
                 [--c-compiler C_COMPILER] [--c++-compiler C++_COMPILER]
                 [--fortran-compiler FORTRAN_COMPILER] [--c-linker C_LINKER]
                 [--c++-linker C++_LINKER] [--fortran-linker FORTRAN_LINKER]
                 [--c-archiver C_ARCHIVER] [--c++-archiver C++_ARCHIVER]
                 [--fortran-archiver FORTRAN_ARCHIVER]
                 [languages [languages ...]]
Positional Arguments
languages

One or more languages that should be configured.

Default: []

Named Arguments
--languages

One or more languages that should be configured.

Default: []

--show-file

Print the path to the config file without updating it.

Default: False

--remove-file

Remove the existing config file and return.

Default: False

--overwrite

Overwrite the existing file.

Default: False

--disable-languages

Possible choices: c, c++, cpp, cxx, fortran, R, matlab, python, r, cmake, make, lpy, osr, sbml, dummy, executable, mpi, timesync

One or more languages that should be disabled.

Default: []

--enable-languages

Possible choices: c, c++, cpp, cxx, fortran, R, matlab, python, r, cmake, make, lpy, osr, sbml, dummy, executable, mpi, timesync

One or more languages that should be enabled.

Default: []

--quiet, -q

Suppress output.

Default: False

--allow-multiple-omp

Have yggdrasil set the environment variable KMP_DUPLICATE_LIB_OK to ‘True’ during model runs to disable errors resembling “OMP: Error #15: Initializing libomp.dylib…” that result from having multiple versions of OpenMP loaded during runtime.

--dont-allow-multiple-omp

Don’t set the KMP_DUPLICATE_LIB_OK environment variable when running models (see help for ‘–allow-multiple-omp’ for more information).

--c-compiler

Name or path to compiler that should be used to compile models written in c.

--c++-compiler

Name or path to compiler that should be used to compile models written in c++.

--fortran-compiler

Name or path to compiler that should be used to compile models written in fortran.

--c-linker

Name or path to linker that should be used to link models written in c.

--c++-linker

Name or path to linker that should be used to link models written in c++.

--fortran-linker

Name or path to linker that should be used to link models written in fortran.

--c-archiver

Name or path to archiver that should be used to create static libraries for models written in c.

--c++-archiver

Name or path to archiver that should be used to create static libraries for models written in c++.

--fortran-archiver

Name or path to archiver that should be used to create static libraries for models written in fortran.

metaschema

Regenerate the yggdrasil metaschema.

yggdrasil metaschema [-h]

schema

Regenerate the yggdrasil schema.

yggdrasil schema [-h] [--only-constants]
Named Arguments
--only-constants

Only update the constants.py file without updating the schema.

Default: False

model-form-schema

Save/print the JSON schema for generating the model specification form.

yggdrasil model-form-schema [-h] [--file FILE]
Named Arguments
--file

Path to file where the schema should be saved.

dev-update

Perform cleanup and reinitialization following an update to the code during development.

yggdrasil dev-update [-h]

test

Run yggdrasil tests.

yggdrasil test [-h] [--withcoverage] [--verbose] [--nocapture] [--stop]
               [--nologcapture] [--noflaky] [--ci]
               [--test-suite {all,top,examples,examples_part1,examples_part2,demos,types,timing,connections,models,mpi} [{all,top,examples,examples_part1,examples_part2,demos,types,timing,connections,models,mpi} ...]]
               [--pytest-config PYTEST_CONFIG] [--cov-config COV_CONFIG]
               [--ignore IGNORE] [--rootdir ROOTDIR] [--count COUNT]
               [--additional-info ADDITIONAL_INFO] [--with-mpi WITH_MPI]
               [--write-script WRITE_SCRIPT]
               [--separate-test SEPARATE_TESTS [SEPARATE_TESTS ...]]
               [--loglevel LOGLEVEL] [--rmq-loglevel RMQ_LOGLEVEL]
               [--client-loglevel CLIENT_LOGLEVEL] [--validate-components]
               [--validate-messages {False,True,First}]
               [--namespace NAMESPACE] [--host HOST] [--vhost VHOST]
               [--user USER] [--password PASSWORD] [--cluster CLUSTER]
               [--with-examples] [--with-demos]
               [--language LANGUAGE [LANGUAGE ...]]
               [--skip-language SKIP_LANGUAGE [SKIP_LANGUAGE ...]]
               [--long-running] [--enable-production-runs]
               [--default-comm DEFAULT_COMM] [--production-run] [--debug]
Named Arguments
--withcoverage, --with-coverage

Record coverage during tests.

Default: False

--verbose, -v

Increase verbosity of output from the test runner

Default: False

--nocapture, -s

Don’t capture output from tests.

Default: False

--stop, -x

Stop after first test failure.

Default: False

--nologcapture

Don’t capture output from log messages generated during tests.

Default: False

--noflaky, --no-flaky

Don’t re-run flaky tests.

Default: False

--ci

Perform addition operations required for testing on continuous integration services.

Default: False

--test-suite, --test-suites

Possible choices: all, top, examples, examples_part1, examples_part2, demos, types, timing, connections, models, mpi

Test suite(s) that should be run.

--pytest-config, -c

Pytest configuration file that should be used instead of the defaults.

--cov-config

Config file for coverage. Default: .coveragerc

--ignore

Ignore path during collection.

--rootdir

Define root directory for tests.

--count

Number of times to repeat a test.

Default: 1

--additional-info, -r

Display additional info for test results.

Default: “”

--with-mpi, --mpi-nproc

Number of MPI processes to run tests on.

Default: 1

--write-script

Name of script that should be created to run tests.

--separate-test, --separate-tests

Flags defining tests that should be run separately.

--loglevel

Logging level for yggdrasil operations.

--rmq-loglevel, --rmqloglevel

Logging level for RabbitMQ operations.

--client-loglevel, --clientloglevel

Logging level for yggdrasil operations on model processes.

--validate-components, --validatecomponents

Validate components on creation using their JSON schema (Decreases performance).

Default: False

--validate-messages, --validatemessages

Possible choices: False, True, First

Which messages should be validated during communication. ‘True’: all messages (decreases performance), ‘False’: no messages, or ‘First’: only the first message a comm sends/receives.

--namespace

RabbitMQ namespace.

--host

RabbitMQ host address.

--vhost

RabbitMQ virtual host address.

--user

RabbitMQ username.

--password

RabbitMQ password.

--cluster

Cluster that should be used.

--with-examples, --enable-examples, --withexamples, --enableexamples

Run example tests when encountered.

Default: False

--with-demos, --enable-demos, --withdemos, --enabledemos

Run demo tests when encountered.

Default: False

--language, --languages

Language(s) that sould not be tested.

Default: []

--skip-language, --skip-languages, --skiplanguage, --skiplanguages

Language(s) that sould not be tested.

Default: []

--long-running, --enable-long, --longrunning, --enablelong

Run long tests when encounterd.

Default: False

--enable-production-runs, --enableproductionruns

Run production level tests when encountered.

Default: False

--default-comm, --defaultcomm

Comm type that should be used by default.

--production-run

Turn off safe guards in order to improve performance. This is equivalent to ‘–validate-components –validate-messages=True’

Default: False

--debug

Turn on debugging utilties including increased logging and validation. This is equivalent to ‘–loglevel=DEBUG –client-loglevel=DEBUG –validate-components=False –validate-messages=False’

Default: False

timing

Create performance plots using timing results.

yggdrasil timing [-h] {commtype,language,os,python,lang2019} ...
comparison

Comparison plot that should be created.

comparison

Possible choices: commtype, language, os, python, lang2019

Sub-commands:
commtype

Compare different communication avenues.

yggdrasil timing commtype [-h]
language

Compare different programming languages.

yggdrasil timing language [-h]
os

Compare different operating systems.

yggdrasil timing os [-h]
python

Compare different versions of Python.

yggdrasil timing python [-h]
lang2019

Create plots from Lang (2019) paper.

yggdrasil timing lang2019 [-h]

gha

Generate a Github Actions (GHA) workflow yaml file from a version of the file that uses anchors (not supported by GHA as of 2021-01-14).

yggdrasil gha [-h] [--base BASE] [--dest DEST] [--verbose]
Named Arguments
--base, --base-file

Version of GHA workflow yaml that contains anchors.

--dest

Name of target GHA workflow yaml file.

--verbose

Print yaml contents.

Default: False

integration-service-manager

Start or manage a integration service manager.

yggdrasil integration-service-manager [-h] [--manager-name MANAGER_NAME]
                                      [--service-type {flask,rmq}]
                                      [--commtype COMMTYPE]
                                      [--address ADDRESS] [--port PORT]
                                      {start,stop,status,register,unregister}
                                      ...
Named Arguments
--manager-name

Name that will be used to identify the service manager.

--service-type

Possible choices: flask, rmq

Type of service that should be started. If not provided, the default will be the (‘services’, ‘default_type’) configuration option, if set, and will otherwise be ‘flask’.

--commtype

Type of communicator that should be used for connections to services. If not provided, the default will be determined by the (‘services’, ‘default_comm’) configuration option, if set.

--address

URL for requests to the service manager. If not provided, the default will be determined by the (‘services’, ‘default_type’) configuration option, if set, and based on the selected ‘service-type’, if not. For a service-type of ‘flask’, this should be the http address with the port that should be used and the default will be ‘http://localhost:{port}’. For a service-type of ‘rmq’, this should be an amqp broker address and the default will be ‘amqp://guest:guest@localhost:{port}/%2f’.

--port

Port that should be used to access the app. Defaults to 5000 for a flask service manager and 5672 for a RabbitMQ service manager if not set via the PORT environment variable.

action

Management action to take.

action

Possible choices: start, stop, status, register, unregister

Sub-commands:
start

Start an integration service manager and/or integration service.

yggdrasil integration-service-manager start [-h] [--remote-url REMOTE_URL]
                                            [--integration-name INTEGRATION_NAME]
                                            [--integration-yamls INTEGRATION_YAMLS [INTEGRATION_YAMLS ...]]
                                            [--with-coverage]
                                            [--model-repository MODEL_REPOSITORY]
                                            [--log-level LOG_LEVEL]
Named Arguments
--remote-url

URL that will be used to access the service manager and running integrations by remote requests. If not provided, the environment variable YGGDRASIL_SERVICE_HOST_URL will be used if it is set. If it is not set, the local address will be used under the assumption that the service manager and integration services will only be connected to by local integrations.

--integration-name

Name of integration to start. If not provided, a service manager will be started.

--integration-yamls

One or more YAML specification files defining the integration. This argument may be omitted if ‘name’ refers to a registered integration.

--with-coverage

Enable coverage cleanup for testing.

Default: False

--model-repository

URL for a directory in a Git repository containing models that should be loaded into the service manager registry.

--log-level

Level of logging that should be performed for the service manager application.

stop

Stop an integration service manager or an integration service.

yggdrasil integration-service-manager stop [-h]
                                           [--integration-name INTEGRATION_NAME]
Named Arguments
--integration-name

Name of integration to stop. If not provided, the service manager will be stopped as will all of the running services.

status

Get list of available services and the status of any running services.

yggdrasil integration-service-manager status [-h]
register

Register an integration with the service manager.

yggdrasil integration-service-manager register [-h]
                                               integration-name
                                               [integration-yamls [integration-yamls ...]]
Positional Arguments
integration-name

The name that the integration should be registered under or the path to a YAML file containing a list of one or more mappings between name and YAML specification files for integrations that should be registered.

integration-yamls

One or more YAML specification files defining the integration.

unregister

Unregister an integration with the service manager.

yggdrasil integration-service-manager unregister [-h] integration-name
Positional Arguments
integration-name

The name of the integration to remove from the registry or the path to a YAML file containing a list of one or more mappings between integration name and YAML specification files for integrations that should be unregistered.

yggcc Command

Alias for yggdrasil compile

yggccflags Command

usage: yggccflags [-h] [--cpp] [--toolname TOOLNAME]

Named Arguments

--cpp

Get the compilation flags used for C++ programs.

Default: False

--toolname

Name of the tool that associated flags be returned for.

yggcctool Command

usage: yggcctool [-h] [--cpp] [--fullpath]

Named Arguments

--cpp

Get the compiler used for C++ programs.

Default: False

--fullpath

Get the full path to the tool exectuable.

Default: False

yggclean Command

Alias for yggdrasil clean

yggcompile Command

Alias for yggdrasil compile-deps

yggconfig Command

Alias for yggdrasil config

yggdevup Command

Alias for yggdrasil dev-update

ygggha Command

Alias for yggdrasil gha

ygginfo Command

Alias for yggdrasil info

ygginstall Command

Alias for yggdrasil install

yggldflags Command

usage: yggldflags [-h] [--cpp] [--toolname TOOLNAME]

Named Arguments

--cpp

Get the compilation flags used for C++ programs.

Default: False

--toolname

Name of the tool that associated flags be returned for.

yggldtool Command

usage: yggldtool [-h] [--cpp] [--fullpath]

Named Arguments

--cpp

Get the linker used for C++ programs.

Default: False

--fullpath

Get the full path to the tool exectuable.

Default: False

yggmetaschema Command

Alias for yggdrasil metaschema

yggmodelform Command

Alias for yggdrasil model-form-schema

yggrun Command

Alias for yggdrasil run

yggschema Command

Alias for yggdrasil schema

yggtest Command

Alias for yggdrasil test

yggvalidate Command

Alias for yggdrasil validate