SIERRA Command Line Extensions

General Options

usage: sierra-cli [--template-input-file filepath] [--exp-overwrite]
                  [--sierra-root dirpath]
                  [--batch-criteria [<category>.<definition>,...]
                  [[<category>.<definition>,...] ...]]
                  [--pipeline [PIPELINE ...]] [--exp-range EXP_RANGE]
                  [--platform-vc] [--processing-serial] [--n-runs N_RUNS]
                  [--skip-collate] [--plot-log-xscale]
                  [--plot-enumerated-xscale] [--plot-log-yscale]
                  [--plot-regression-lines PLOT_REGRESSION_LINES]
                  [--plot-primary-axis PLOT_PRIMARY_AXIS] [--plot-large-text]
                  [--plot-transpose-graphs] [--scenario <block dist>.AxBxC]
                  [--controller {d0, d1, d2}.<controller>]

Multi-stage options

Options which are used in multiple pipeline stages

--template-input-file

The template .xml input file for the batch experiment. Beyond the requirements for the specific --platform, the content of the file can be any valid XML, with the exception of the SIERRA requirements detailed in ln-sierra-tutorials-project-template-input-file.

Tip

Used by stage {1,2,3,4}; can be omitted otherwise. If omitted: N/A.

--exp-overwrite

When SIERRA calculates the batch experiment root (or any child path in the batch experiment root) during stage {1, 2}, if the calculated path already exists it is treated as a fatal error and no modifications to the filesystem are performed. This flag overwrides the default behavior. Provided to avoid accidentally overwrite input/output files for an experiment, forcing the user to be explicit with potentially dangerous actions.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

Default: False

--sierra-root

Root directory for all SIERRA generated and created files.

Subdirectories for controllers, scenarios, experiment/experimental run inputs/outputs will be created in this directory as needed. Can persist between invocations of SIERRA.

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

Default: “<home directory>/exp”

--batch-criteria

Definition of criteria(s) to use to define the experiment.

Specified as a list of 0 or 1 space separated strings, each with the following general structure:

<category>.<definition>

<category> must be a filename from the core/variables/ or from a --project <project>/variables/ directory, and <definition> must be a parsable namne (according to the requirements of the criteria defined by the parser for <category>).

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. If omitted: N/A.

Default: []

--pipeline

Define which stages of the experimental pipeline to run:

Stage1 - Generate the experiment definition from the template input file, batch criteria, and other command line options. Part of default pipeline.

Stage2 - Run a previously generated experiment. Part of default pipeline.

Stage3 - Post-process experimental results after running the batch experiment; some parts of this can be done in parallel. Part of default pipeline.

Stage4 - Perform deliverable generation after processing results for a batch experiment, which can include shiny graphs and videos. Part of default pipeline.

Stage5 - Perform graph generation for comparing controllers AFTER graph generation for batch experiments has been run. Not part of default pipeline.

Default: [1, 2, 3, 4]

--exp-range

Set the experiment numbers from the batch to run, average, generate intra-experiment graphs from, or generate inter-experiment graphs from (0 based). Specified in the form min_exp_num:max_exp_num (closed interval/inclusive). If omitted, runs, averages, and generates intra-experiment and inter-experiment performance measure graphs for all experiments in the batch (default behavior).

This is useful to re-run part of a batch experiment in HPC environments if SIERRA gets killed before it finishes running all experiments in the batch, or to redo a single experiment with real robots which failed for some reason.

Tip

Used by stage {2,3,4}; can be omitted otherwise. If omitted: N/A.

--platform-vc

For applicable --platforms, enable visual capturing of run-time data during stage 2. This data can be frames (i.e., .png files), or rendering videos, depending on the platform. If the captured data was frames, then SIERRA can render the captured frames into videos during stage 4. If the selected --platform does not support visual capture, then this option has no effect. See ln-sierra-usage-rendering-platform for full details.

Tip

Used by stage {1,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--processing-serial

If TRUE, then results processing/graph generation will be performed serially, rather than using parallellism where possible.

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--n-runs

The # of experimental runs that will be executed and their results processed to form the result of a single experiment within a batch.

If --platform is a simulator and --exec-env is something other than hpc.local then this may be be used to determine the concurrency of experimental runs.

Tip

Used by stage {1,2}; can be omitted otherwise. If omitted: N/A.

--skip-collate

Specify that no collation of data across experiments within a batch (stage 4) or across runs within an experiment (stage 3) should be performed. Useful if collation takes a long time and multiple types of stage 4 outputs are desired. Collation is generally idempotent unless you change the stage3 options (YMMV).

Tip

Used by stage {3,4}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-log-xscale

Place the set of X values used to generate intra- and inter-experiment graphs into the logarithmic space. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

SummaryLineGraph

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-enumerated-xscale

Instead of using the values generated by a given batch criteria for the X values, use an enumerated list[0, …, len(X value) - 1]. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

SummaryLineGraph

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-log-yscale

Place the set of Y values used to generate intra - and inter-experiment graphs into the logarithmic space. Mainly useful when the batch criteria involves large system sizes, so that the plots are more readable.

Tip

Applicable graphs:

SummaryLineGraph
StackedLineGraph

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-regression-lines

For all 2D generated scatterplots, plot a linear regression line and the equation of the line to the legend.

Tip

Applicable graphs:

SummaryLineGraph

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

--plot-primary-axis

This option allows you to override the primary axis, which is normally is computed based on the batch criteria.

For example, in a bivariate batch criteria composed of

ln-sierra-platform-argos-bc-population-size on the X axis (rows)

Another batch criteria which does not affect system size (columns)

Metrics will be calculated by computing across .csv rows and projecting down the columns by default, since system size will only vary within a row. Passing a value of 1 to this option will override this calculation, which can be useful in bivariate batch criteria in which you are interested in the effect of the OTHER non-size criteria on various performance measures.

0=criteria of interest varies across rows.

1=criteria of interest varies across columns.

This option only affects generating graphs from bivariate batch criteria.

Tip

Applicable graphs:

Heatmap
StackedLineGraph

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

--plot-large-text

This option specifies that the title, X/Y axis labels/tick labels should be larger than the SIERRA default. This is useful when generating graphs suitable for two column paper format where the default text size for rendered graphs will be too small to see easily. The SIERRA defaults are generally fine for the one column/journal paper format.

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--plot-transpose-graphs

Transpose the X, Y axes in generated graphs. Useful as a general way to tweak graphs for best use of space within a paper.

Changed in version 1.2.20: Renamed from --transpose-graphs to make its relation to other plotting options clearer.

Tip

Applicable graphs:

Heatmap

Tip

Used by stage {4,5}; can be omitted otherwise. If omitted: N/A.

Default: False

--scenario

Which scenario the swarm comprised of robots running the controller specified via --controller should be run in.

A scenario is defined as: block distribution type + arena dimensions. This is somewhat tied to foraging and other similar applications for the moment, but this may be modified in a future version of TITERRA.

Valid block distribution types:

RN - Random

SS - Single source

DS - Dual source

QS - Quad source

PL - Power law

A,B,C are the scenario X,Y,Z dimensions respectively (which can be any postive INTEGER values). All dimensions are required.

Tip

Used by stage {1,2,3,4}; can be omitted otherwise. If omitted: N/A.

--controller

Possible choices: d0.CRW, d0.DPO, d0.ODPO, d0.MDPO, d0.OMDPO, d1.BITD_DPO, d1.BITD_ODPO, d1.BITD_OMDPO, d2.BIRTD_DPO, d2.BIRTD_ODPO, d2.BIRTD_OMDPO

Which controller robots will use in the foraging experiment. All robots use the same controller (homogeneous swarms).

Head over to the FORDYCA docs for the descriptions of these controllers.

Tip

Used by stage {1,2,3,4,5}; can be omitted otherwise. Only required for stage 5 if --scenario-comp is passed.

Stage1: Generating Experiments

usage: sierra-cli [--preserve-seeds PRESERVE_SEEDS] [--no-preserve-seeds]
                  [--n-blocks N_BLOCKS]
                  [--static-cache-blocks STATIC_CACHE_BLOCKS]

Stage1: General options for generating experiments

--preserve-seeds: Preserve previously generated random seeds for experiments (the default). Useful for regenerating experiments when you change parameters/python code that don’t affects experimental outputs (e.g., paths). Preserving/overwriting random seeds is not affected by --exp-overwrite.

Default: True
--no-preserve-seeds: Opposite of --preserve-seeds.

Default: True
--n-blocks: # blocks that should be used in the simulation. Can be used to override batch criteria, or to supplement experiments that do not set it so that manual modification of input file is unneccesary.

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.
--static-cache-blocks: # of blocks used when the static cache is # respawned (d1 controllers only).

Tip

Used by stage {1}; can be omitted otherwise. If omitted: N/A.