Part 1: CLI Reference

The doe command-line interface provides 23 subcommands for generating, running, analyzing, and managing experimental designs. All commands follow the pattern:

Terminal

$ doe <command> [options]

ℹ

Global Behavior

The --config flag accepts any valid JSON file path. Relative paths are resolved from the current working directory. Most commands that read results use the out_directory field from the config unless overridden with --results-dir.

`doe generate`

Generate a design matrix and write a runner script that executes all experimental runs.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file defining factors, responses, and design settings.
`--output PATH`	string	`run_experiments.sh`	Output path for the generated runner script.
`--format {sh,py}`	choice	`sh`	Script format. `sh` produces a Bash script; `py` produces a Python script.
`--seed INT`	integer	none	Random seed for run-order randomization. If omitted, run order is not randomized.
`--dry-run`	flag	off	Print the design matrix to stdout without writing any files.
`--session [PREFIX]`	string	off	Each runner invocation writes results into a fresh `<out>/<PREFIX>-<TIMESTAMP>/` directory and updates the `<out>/latest` symlink.
`--resolution INT`	integer	none	Fractional factorial only: bump the run count until the design reaches at least Resolution N.
`--replicate-center INT`	integer	none	Append N center-point runs to each block for a pure-error estimate.
`--parallel INT`	integer	`1`	Emit a thread-pool Python runner that executes N runs concurrently.
`--executor {local,slurm}`	choice	`local`	`slurm` emits an `sbatch --array` script. Tune it with `--slurm-partition`, `--slurm-time`, `--slurm-cpus-per-task`, `--slurm-mem`, and `--slurm-max-concurrent`.

Example

$ doe generate --config experiment.json --seed 42 --format py
$ doe generate --config experiment.json --dry-run

`doe analyze`

Analyze completed experiment results. Computes effects, ANOVA, generates plots, and produces an HTML report.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--results-dir DIR`	string	from config	Override the `out_directory` setting from the config file.
`--no-plots`	flag	off	Skip generating Pareto, main effects, interaction, and diagnostic plots.
`--no-report`	flag	off	Skip generating the HTML analysis report.
`--csv DIR`	string	none	Export analysis results (effects, ANOVA, summaries) as CSV files into `DIR`.
`--partial`	flag	off	Analyze only the runs that have completed so far (ignore missing results).
`--knee`	flag	off	Detect saturation/knee points in response curves where diminishing returns begin.
`--factor NAME [NAME...]`	string(s)	all	Restrict analysis to the specified factor(s) only.
`--filter-runs N [N...]`	integer(s)	none	Exclude the given run IDs (e.g., known outliers) from the analysis without editing result files.
`--no-rsm`	flag	off	Skip the quadratic RSM refit and the model-adequacy, stationary-point, and cross-validation sections. Useful for large designs.
`--cv-folds INT`	integer	`min(n, 5)`	Cross-validation fold count for the response surface model. Pass `n` for leave-one-out.

Example

$ doe analyze --config experiment.json --csv exports/
$ doe analyze --config experiment.json --partial --knee --factor temperature pressure

`doe info`

Display summary information about a design: number of factors, levels, design type, run count, and design evaluation metrics.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.

`doe optimize`

Find optimal factor settings from experiment results. Supports single-response optimization, multi-objective desirability, and steepest ascent/descent pathways.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--response NAME`	string	all	Optimize for a specific response variable. By default, considers all responses.
`--partial`	flag	off	Use only completed runs for optimization.
`--multi`	flag	off	Multi-objective optimization using desirability functions. Requires `bounds` on responses.
`--steepest`	flag	off	Show steepest ascent (for maximize) or descent (for minimize) pathway from the current best point.

`doe report`

Generate a standalone HTML analysis report without running the full analysis pipeline.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--output PATH`	string	`report.html`	Output path for the HTML report file.
`--partial`	flag	off	Generate a report using only completed runs.
`--include FILE`	string (repeatable)	none	Inline another HTML file (e.g., a `compare`, `trend`, or `sensitivity` page) as an extra report section. Repeat for multiple files.

`doe record`

Manually record response values for one or all runs. Prompts interactively for each response value.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--run {N\|all}`	string	required	Run number (1-based) to record, or `all` to iterate through all pending runs.
`--seed INT`	integer	`42`	Random seed used to determine run order (must match the seed used in `generate`).

`doe status`

Show experiment progress: how many runs are completed, pending, and failed.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--seed INT`	integer	`42`	Random seed used to determine run order.

`doe power`

Perform power analysis to determine whether the design has enough runs to detect a given effect size at the specified significance level.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--sigma FLOAT`	float	estimated	Error standard deviation. If omitted and results exist, estimated from residuals.
`--delta FLOAT`	float	none	Minimum detectable effect size (the smallest effect you care about).
`--alpha FLOAT`	float	`0.05`	Significance level (Type I error rate).
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--partial`	flag	off	Use only completed runs when estimating sigma from results.

`doe augment`

Augment an existing design with additional runs. Use to de-alias effects (fold-over), fit quadratic models (star points), or estimate pure error (center points).

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--type {fold_over,star_points,center_points,d_optimal}`	choice	required	Augmentation type. `fold_over` reverses all signs; `star_points` adds axial points; `center_points` adds center-point replicates; `d_optimal` adds the runs that most improve D-efficiency.
`--output PATH`	string	`run_experiments_augmented.sh`	Output path for the augmented runner script.
`--format {sh,py}`	choice	`sh`	Script format for the augmented runner.
`--seed INT`	integer	none	Random seed for run-order randomization of augmented runs.
`--session [PREFIX]`	string	off	Write augmented-run results into a fresh timestamped session directory (same semantics as `doe generate --session`).

`doe init`

Initialize a new experiment from a use-case template. Creates a config file and optionally a test script scaffold.

Flag	Type	Default	Description
`--template NAME`	string	none	Use-case template name (e.g., `web-server`, `database`, `ml-hyperparameter`).
`--list`	flag	off	List all available templates with descriptions.
`--output-dir DIR`	string	`.`	Directory where the config file and scaffold will be written.
`--factors INT`	integer	none	Bootstrap a working config from scratch (no template) with this many factors. Requires `--budget`.
`--responses INT`	integer	`1`	Number of response variables when bootstrapping.
`--budget INT`	integer	none	Run budget when bootstrapping; the design type is chosen to fit it. Required with `--factors`.
`--goal {screening,response_surface,optimization}`	choice	`screening`	Experimental goal used to pick the design when bootstrapping.
`--categorical INT`	integer	`0`	Number of categorical factors when bootstrapping.
`--with-test`	flag	off	Also scaffold a `test.py` beside the new config.

Example

$ doe init --list
$ doe init --template web-server --output-dir my-experiment/
$ doe init --factors 4 --budget 20 --goal screening --with-test

`doe export-worksheet`

Export a blank worksheet showing the design matrix with columns for manually recording responses.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--format {csv,markdown}`	choice	`csv`	Output format: CSV or Markdown table.
`--output FILE`	string	stdout	Output file path. Defaults to printing to stdout.
`--seed INT`	integer	`42`	Random seed for run order.

`doe export-data`

Export the design matrix and collected response values as a flat data file for use in external analysis tools.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--format {csv,tsv}`	choice	`csv`	Output format: comma-separated or tab-separated.
`--output FILE`	string	stdout	Output file path. Defaults to printing to stdout.
`--seed INT`	integer	`42`	Random seed for run order.
`--partial`	flag	off	Include only completed runs (skip missing results).

`doe next-batch`

Generate the next batch of runs for sequential experimentation. Analyzes current results and selects new points based on the chosen strategy.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--strategy {refine,explore,balanced,model_guided,bayesian,multi_objective}`	choice	from config	`refine`: focus near the best region. `explore`: maximize space coverage. `balanced`: mix of both. `model_guided`: place runs where the fitted surface is most uncertain. `bayesian`: Gaussian-process expected improvement. `multi_objective`: desirability-weighted acquisition across responses.
`--batch-size INT`	integer	auto	Number of new runs to generate. Auto-selects based on factor count if omitted.
`--output PATH`	string	`run_next_batch.sh`	Output path for the batch runner script.
`--format {sh,py}`	choice	`sh`	Script format for the batch runner.
`--seed INT`	integer	none	Random seed for run-order randomization.
`--partial`	flag	off	Consider only completed runs when planning the next batch.
`--session [PREFIX]`	string	off	Write batch results into a fresh timestamped session directory.
`--state-name NAME`	string	none	Branch the adaptive trajectory by writing phase state under a named key, so alternative follow-up paths can be explored from the same starting data.

`doe suggest`

Recommend a design operation, run count, and adaptive strategy for your situation — before writing any config. Answers “which design should I use?” from the factor count, response count, run budget, and goal.

Flag	Type	Default	Description
`--factors INT`	integer	required	Number of factors you plan to vary.
`--responses INT`	integer	`1`	Number of response variables you will measure.
`--budget INT`	integer	required	Maximum number of runs you can afford.
`--goal {screening,response_surface,optimization}`	choice	`screening`	What you want out of the experiment.
`--categorical INT`	integer	`0`	How many of the factors are categorical.

Example

$ doe suggest --factors 5 --budget 24 --goal screening

`doe scaffold-config`

Write an annotated starter config.json with sample factors, responses, and option hints to edit into your own experiment.

Flag	Type	Default	Description
`--output FILE`	string	`config.json`	Where to write the starter config.
`--force`	flag	off	Overwrite the output file if it already exists.

`doe scaffold-test`

Write a starter test.py or test.sh that already parses the factors from your config using the configured arg_style and emits a valid result JSON — you only fill in the measurement.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--language {py,sh}`	choice	`py`	Scaffold language.
`--output FILE`	string	auto	Output path. Defaults to `test.py` / `test.sh`.
`--force`	flag	off	Overwrite the output file if it already exists.

`doe simulate`

Evaluate the design directly against a Python function — no runner script, no subprocesses. Each run's factor values are passed as keyword arguments and the function's return dict becomes the result JSON.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--func TARGET`	string	required	Python target as `module:function` or `path/to/file.py:function`.
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--session [PREFIX]`	string	off	Write results into a fresh timestamped session directory.
`--overwrite`	flag	off	Re-evaluate runs whose result file already exists.
`--seed INT`	integer	`42`	Random seed for run order.

Example

$ doe simulate --config experiment.json --func my_model.py:evaluate

`doe compare`

Pairwise comparison of two sessions: per-run deltas with a paired t-test and Cohen's d, per-factor effect deltas with sign-flip flags, and an intercept-shift vs slope-shift decomposition.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--baseline DIR`	string	required	Results directory of the baseline session.
`--candidate DIR`	string	required	Results directory of the candidate session to compare against the baseline.
`--csv DIR`	string	none	Also export the comparison tables as CSV files into `DIR`.
`--html FILE`	string	none	Also write a self-contained HTML comparison page.

Example

$ doe compare --config experiment.json --baseline results/before-tune --candidate results/after-tune --html compare.html

`doe trend`

Multi-session regression across three or more sessions: per-session response means plus intercept and slope drift per session step. Use it to watch a system change over repeated experiments.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--sessions DIR [DIR...]`	string(s)	required	Two or more session result directories, in chronological order.
`--csv DIR`	string	none	Also export the trend tables as CSV files into `DIR`.
`--html FILE`	string	none	Also write a self-contained HTML trend page.

`doe sensitivity`

Global sensitivity analysis on the fitted response surface: Sobol first-order and total-order indices per factor, with optional stacked-bar HTML visualizations.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--results-dir DIR`	string	from config	Override the `out_directory` setting.
`--response NAME`	string	all	Analyze a specific response variable only.
`--n-samples INT`	integer	`512`	Number of Sobol samples per index estimate.
`--csv FILE`	string	none	Optional CSV output path for the index table.
`--html FILE`	string	none	Optional self-contained HTML report with stacked-bar plots.
`--seed INT`	integer	`42`	Random seed for Sobol sampling.
`--partial`	flag	off	Analyze only completed runs, skipping missing results.

`doe calibrate`

Fit the free parameters of a parametric Python simulator so that its output matches observed experimental data. The calibrated simulator can then stand in for expensive real runs.

Flag	Type	Default	Description
`--config FILE`	string	required	Input JSON configuration file.
`--func TARGET`	string	required	Parametric simulator as `module:function` or `path.py:function`.
`--params SPEC [SPEC...]`	string(s)	required	Parameter specs as `name:low:high` or `name:initial:low:high`.
`--observed DIR`	string	required	Results directory containing the observed run data to fit against.
`--report FILE`	string	none	Optional path to write the JSON calibration report.
`--seed INT`	integer	`42`	Random seed.

Example

$ doe calibrate --config experiment.json --func sim.py:reactor --params k1:0.1:2.0 ea:40:120 --observed results/latest

`doe archive`

Bundle a session into a single .tar.gz with a SHA-256 manifest for sharing or long-term storage. Optionally embeds the config and extra files alongside the results.

Flag	Type	Default	Description
`--session DIR`	string	required	Session results directory to archive.
`--output FILE`	string	required	Output archive path (e.g., `experiment.tar.gz`).
`--config FILE`	string	none	Optional config file to embed alongside the session.
`--extra FILE`	string (repeatable)	none	Additional files to include. Repeat for multiple files.

`doe serve`

Start a localhost HTTP server (Python stdlib only) that lists sessions under a results root and links to their HTML reports for quick browsing.

Flag	Type	Default	Description
`--root DIR`	string	`results`	Root directory containing session subdirectories.
`--host HOST`	string	`127.0.0.1`	Interface to bind.
`--port INT`	integer	`8000`	Port to listen on.

Part 2: Configuration Schema

The configuration file (config.json) is a single JSON object that defines the entire experiment. Every doe command reads this file.

Top-level Fields

Field	Type	Required	Description
`metadata`	object	No	Contains `name` (string) and `description` (string) for labeling the experiment.
`factors`	array of Factor	Yes	Array of factor definitions. Must contain at least 1 factor (2+ for most design types).
`fixed_factors`	object	No	Key-value pairs passed to every run but not varied. Useful for constants.
`responses`	array of ResponseVar	No	Response variables to collect and analyze. If omitted, all keys in result JSON are treated as responses.
`runner`	RunnerConfig	No	Controls how factor values are passed to the test script.
`settings`	object	No	Design type, blocking, output directories, and other operational settings.
`constraints`	array of strings	No	Boolean expressions over factor names (e.g., `"temperature * pressure <= 400"`). Candidate runs violating any constraint are excluded. Expressions are parsed against a safe AST allow-list — no arbitrary code execution.
`adaptive`	object	No	Sequential experimentation settings for `doe next-batch`: `strategy` (one of `refine`, `explore`, `balanced`, `model_guided`, `bayesian`, `multi_objective`; default `refine`), `batch_size` (default `4`), `response_name`, and stopping rules `stopping_effect_threshold`, `stopping_power_threshold`, `stopping_max_phases` (default `10`).

Minimal config.json

{
  "metadata": { "name": "My Experiment", "description": "Testing two factors" },
  "factors": [
    { "name": "temperature", "levels": ["60", "80"], "type": "continuous" },
    { "name": "pressure", "levels": ["1.0", "2.0"], "type": "continuous" }
  ],
  "responses": [
    { "name": "yield", "optimize": "maximize" }
  ],
  "settings": {
    "operation": "full_factorial",
    "test_script": "./run_test.sh",
    "out_directory": "results"
  }
}

Factor Object

Field	Type	Default	Description
`name`	string	required	Unique identifier for the factor. Used as the argument name in runner scripts and as column headers in output.
`levels`	array of strings	required	At least 2 values. For continuous factors, these define the low and high (and optionally center) points. Values are always strings in the config.
`type`	string	`"categorical"`	`"categorical"` — unordered discrete values. `"continuous"` — numeric range (enables RSM, CCD, etc.). `"ordinal"` — ordered discrete values (enables trend analysis).
`unit`	string	none	Display unit (e.g., `"C"`, `"MB"`, `"ms"`). Shown in reports and plots.
`description`	string	none	Human-readable description of the factor.
`dtype`	string	`""` (auto)	Force a data type for level values: `"int"`, `"float"`, or `""` for automatic detection. `"int"` values are rounded and clamped by all optimizers.
`role`	string	`"subplot"`	Split-plot designs only: `"whole_plot"` marks a hard-to-change factor that stays fixed within each whole plot; `"subplot"` factors vary within plots. Ignored by other design types.

⚠

Level Values Are Strings

All level values in the JSON config must be strings, even for numeric factors. The tool parses them based on the type and dtype fields. Example: "levels": ["100", "200"], not "levels": [100, 200].

ResponseVar Object

Field	Type	Default	Description
`name`	string	required	Must match a key in the result JSON output by the test script.
`optimize`	string	`"maximize"`	`"maximize"` or `"minimize"`. Determines optimization direction and steepest ascent/descent.
`unit`	string	none	Display unit shown in reports and plots.
`description`	string	none	Human-readable description.
`weight`	float	`1.0`	Relative importance weight for multi-objective optimization. Higher weight gives this response more influence on the overall desirability.
`bounds`	[float, float]	none	Two-element array: `[worst, best]`. Required for desirability-based multi-objective optimization (`--multi`). For maximize: worst < best. For minimize: worst > best.

RunnerConfig

Field	Type	Default	Description
`arg_style`	string	`"double-dash"`	How factor values are passed to the test script: `"double-dash"` — `./test.sh --temperature 80 --pressure 2.0` `"env"` — `TEMPERATURE=80 PRESSURE=2.0 ./test.sh` `"positional"` — `./test.sh 80 2.0` (in factor definition order)
`result_file`	string	`"json"`	Expected output format from the test script. Currently `"json"` is the supported format.

Settings

Field	Type	Default	Description
`operation`	string	`"full_factorial"`	Design type. One of the 14 supported design types (see Part 3).
`test_script`	string	none	Path to the test executable. Invoked for each experimental run.
`block_count`	integer	`1`	Number of blocks (replicates) for the design. Values > 1 replicate the entire design matrix.
`out_directory`	string	`"results"`	Directory where result JSON files are stored, one per run.
`processed_directory`	string	none	Directory to move result files after processing. If omitted, results stay in `out_directory`.
`lhs_samples`	integer	`0`	Number of sample points for Latin Hypercube designs. `0` = auto: `max(10, 2 * n_factors)`.
`sweep_points`	integer	`0`	Number of points for linear/log sweep designs. `0` = auto.
`min_resolution`	integer	`0`	Fractional factorial only: minimum design resolution. The run count is bumped until the resolution is reached. Equivalent to the `--resolution` flag.
`replicate_center`	integer	`0`	Append this many center-point runs to each block for a pure-error estimate. Equivalent to the `--replicate-center` flag.
`whole_plot_replicates`	integer	`1`	Split-plot designs only: number of times each whole-plot setting group is replicated.

Part 3: Design Types

The operation field in settings selects the experimental design. Each design type has different requirements, run counts, and estimation capabilities.

full_factorial

Full Factorial Design

Property	Value
Operation string	`"full_factorial"`
Purpose	Evaluate all possible combinations of factor levels. The most thorough design — no aliasing, complete information.
Requirements	1+ factors, 2+ levels each. No restrictions on factor types.
Run count	L₁ × L₂ × ... × L_k (product of all level counts)
Algorithm	Cartesian product of all factor levels. Optionally randomized and/or blocked.
Estimates	All main effects, all 2-factor interactions, all higher-order interactions up to k-way.

ℹ

Run count grows exponentially: a 2⁵ design has 32 runs, but a 3⁵ design has 243. Consider fractional or screening designs when k > 4 with 3+ levels.

fractional_factorial

Fractional Factorial Design

Property	Value
Operation string	`"fractional_factorial"`
Purpose	Estimate main effects and some interactions with a fraction of the full factorial runs. Trades resolution for efficiency.
Requirements	2+ factors. Exactly 2 levels per factor.
Run count	2^k−p, where p is chosen automatically to achieve Resolution III or higher.
Algorithm	Uses generators to define the fraction. The defining relation determines which effects are aliased (confounded).
Estimates	Main effects (aliased with higher-order interactions at Resolution III). At Resolution IV+, main effects are clear of 2-factor interactions.

plackett_burman

Plackett-Burman Design

Property	Value
Operation string	`"plackett_burman"`
Purpose	Screening: identify which factors have significant main effects with minimal runs. Resolution III.
Requirements	2+ factors. Exactly 2 levels per factor.
Run count	N = smallest multiple of 4 that is ≥ k + 1 (e.g., 12 runs for up to 11 factors).
Algorithm	Constructs a Hadamard-like matrix. Extra columns beyond k factors become dummy factors for error estimation.
Estimates	Main effects only. All 2-factor interactions are partially aliased with main effects.

latin_hypercube

Latin Hypercube Sampling

Property	Value
Operation string	`"latin_hypercube"`
Purpose	Space-filling design for exploring a continuous factor space. Good for computer experiments and metamodeling.
Requirements	1+ continuous factors with exactly 2 levels (defining the range endpoints).
Run count	Configurable via `lhs_samples`. Default: max(10, 2k).
Algorithm	Divides each factor range into n equal strata and places exactly one sample in each stratum per factor. Optimized for maximin distance criterion.
Estimates	No formal aliasing structure. Provides a response surface approximation via regression or interpolation. Not orthogonal.

central_composite

Central Composite Design (CCD)

Property	Value
Operation string	`"central_composite"`
Purpose	Response Surface Methodology (RSM). Fits a full second-order (quadratic) model to find optima.
Requirements	2+ continuous factors with exactly 2 levels.
Run count	2^k + 2k + n_c (factorial points + star points + center points). For k=3: 8 + 6 + 6 = 20.
Algorithm	Combines a factorial core (2^k), axial/star points at distance α along each axis, and center-point replicates. The α value is set to make the design rotatable (α = 2^k/4).
Estimates	All main effects, all 2-factor interactions, all quadratic (squared) terms. Full second-order model.

box_behnken

Box-Behnken Design

Property	Value
Operation string	`"box_behnken"`
Purpose	RSM design that avoids extreme corner points. Useful when corners are infeasible or dangerous.
Requirements	3+ continuous factors with exactly 2 levels.
Run count	Depends on k. For k=3: 12 + center points. For k=4: 24 + center points. Generally fewer runs than CCD.
Algorithm	Combines 2² factorials for each pair of factors while holding remaining factors at the center level. Adds center-point replicates.
Estimates	All main effects, all 2-factor interactions, all quadratic terms. Full second-order model.

definitive_screening

Definitive Screening Design (DSD)

Property	Value
Operation string	`"definitive_screening"`
Purpose	Modern screening design that can detect curvature and some 2-factor interactions, unlike traditional screening designs.
Requirements	3+ factors with exactly 2 levels (continuous). Internally uses 3 levels: low, center, high.
Run count	2k + 1 runs for k factors (e.g., 13 runs for 6 factors).
Algorithm	Conference matrix construction. Each pair of columns has the property that when one factor is at its center, the other varies across all three levels.
Estimates	Main effects (unaliased with 2-factor interactions). Quadratic effects (detects curvature). Some 2-factor interactions (with k ≥ 6).

taguchi

Taguchi Orthogonal Array

Property	Value
Operation string	`"taguchi"`
Purpose	Robust design methodology. Optimizes signal-to-noise (S/N) ratios to find settings that are robust to noise factors.
Requirements	2+ factors, 2+ levels each. Automatically selects the appropriate orthogonal array (L4, L8, L9, L12, L16, L18, L27, etc.).
Run count	Determined by the selected orthogonal array. Depends on factor count and level count.
Algorithm	Selects the smallest standard orthogonal array that can accommodate all factors. Assigns factors to array columns.
Estimates	Main effects. S/N ratios (larger-is-better, smaller-is-better, nominal-is-best). Limited interaction estimation depending on the array.

d_optimal

D-Optimal Design

Property	Value
Operation string	`"d_optimal"`
Purpose	Algorithmically constructed design that maximizes the determinant of the information matrix. Useful for irregular design spaces and custom run budgets.
Requirements	2+ factors, 2+ levels each. Works with any combination of factor types.
Run count	User-specified or auto-calculated. Must be at least p (number of model parameters).
Algorithm	Fedorov coordinate-exchange algorithm. Starts from a candidate set of all possible combinations, iteratively swaps points to maximize \|X'X\|.
Estimates	Main effects and interactions (depending on the model specified). Optimized for the assumed model.

mixture_simplex_lattice

Mixture Simplex-Lattice Design

Property	Value
Operation string	`"mixture_simplex_lattice"`
Purpose	Formulation experiments where components must sum to a constant (e.g., 1 or 100%). Explores the simplex design space on a regular lattice grid.
Requirements	2+ factors (mixture components). Levels define proportion increments.
Run count	C(q + m - 1, m) where q = number of components, m = lattice degree.
Algorithm	Generates all lattice points on the simplex where each component takes values 0, 1/m, 2/m, ..., 1 and all components sum to 1.
Estimates	Linear and interaction blending effects in Scheffé polynomial models.

mixture_simplex_centroid

Mixture Simplex-Centroid Design

Property	Value
Operation string	`"mixture_simplex_centroid"`
Purpose	Formulation experiments. Uses centroid points of the simplex for a more focused exploration than the lattice design.
Requirements	2+ factors (mixture components).
Run count	2^q - 1 points (vertices, edge midpoints, face centroids, overall centroid).
Algorithm	Generates all subsets of components: pure components (vertices), binary blends (edge midpoints), ternary blends (face centroids), up to the overall centroid where all components are equal.
Estimates	Full Scheffé polynomial up to the q-th degree blending terms.

linear_sweep

Linear Parameter Sweep

Property	Value
Operation string	`"linear_sweep"`
Purpose	Characterize a single factor's effect across its range with evenly spaced points.
Requirements	Exactly 1 continuous factor with 2 levels (defining min and max).
Run count	Configurable via `sweep_points`. Default: auto.
Algorithm	Generates linearly spaced points from the low level to the high level.
Estimates	Response curve shape. Enables knee-point detection and saturation analysis.

log_sweep

Logarithmic Parameter Sweep

Property	Value
Operation string	`"log_sweep"`
Purpose	Characterize a single factor across a wide dynamic range with logarithmically spaced points.
Requirements	Exactly 1 continuous factor with 2 levels (both positive). Defines min and max on a log scale.
Run count	Configurable via `sweep_points`. Default: auto.
Algorithm	Generates points spaced evenly on a log₁₀ scale between the low and high levels.
Estimates	Response curve shape on a logarithmic axis. Useful for parameters spanning orders of magnitude.

split_plot

Split-Plot Design

Property	Value
Operation string	`"split_plot"`
Purpose	Experiments where some factors are hard or expensive to change (oven temperature, machine setup) and others are easy. Hard-to-change factors are held fixed within each whole plot while easy factors vary inside it.
Requirements	At least one factor with `"role": "whole_plot"` and at least one subplot factor.
Run count	(whole-plot level combinations × `whole_plot_replicates`) × subplot combinations.
Algorithm	Crosses the whole-plot factor levels, replicates each whole plot per `whole_plot_replicates`, then crosses the subplot factors within each plot. Runs carry a `whole_plot_id` so analysis can separate the two error strata.
Estimates	Whole-plot and subplot effects tested against their own error terms — ANOVA automatically uses the split-plot error structure instead of a single pooled residual.

ℹ

Choosing a Design

Start with full_factorial if you have ≤ 4 factors with 2 levels. Use plackett_burman or definitive_screening to screen many factors. Move to central_composite or box_behnken for optimization. Use latin_hypercube for space-filling exploration of continuous spaces.

Part 4: Analysis Output

The doe analyze command produces statistical tables, effect estimates, and diagnostic metrics. This section documents every output structure.

ANOVA Table

The Analysis of Variance table decomposes total variability into components attributable to each factor and error.

Column	Description
Source	Factor name, interaction term (e.g., `A:B`), or `Error` / `Residual`.
DF	Degrees of freedom. For a factor with L levels: DF = L - 1. For interactions: product of individual DFs. Error DF = N - (total model DF) - 1.
SS	Sum of squares. The portion of total variability explained by this source.
MS	Mean square = SS / DF.
F	F-statistic = MS_source / MS_error. Measures the ratio of explained to unexplained variance.
p-value	Probability of observing the F-statistic under the null hypothesis. Values < 0.05 are typically considered significant.

Error Estimation Methods

When there are no replicates, error must be estimated by alternative methods:

Method	When Used	Description
Replicate error	block_count > 1 or center points present	Pure error from replicated runs. The gold standard.
Pooled higher-order	Full/fractional factorials without replicates	Pools the smallest effects (assumed negligible) into an error term. Uses the sum of squares from the smallest 1/3 of interaction terms.
Lenth's PSE	Unreplicated 2-level designs	Pseudo Standard Error based on the median of absolute effect estimates. Robust to active effects.

Effect Estimates

For each factor and response, the analysis reports:

Output	Description
Main effect magnitude	For 2-level factors: mean(high) - mean(low). For multi-level: max(level means) - min(level means). See formulas.
Standard error (SE)	Estimated standard error of the effect. Computed from MS_error and the design matrix.
Confidence interval	Effect ± t_{α/2, df} × SE. Default α = 0.05 (95% CI).
Contribution %	100 × SS_factor / SS_total. Percentage of total variability attributable to this factor.
Significance	p-value from the F-test. Marked as significant if p < 0.05.

Interaction Effects

Two-factor interaction effects measure how the effect of one factor depends on the level of another. Reported for designs with sufficient degrees of freedom (full factorial, CCD, Box-Behnken, etc.).

Interaction magnitude: half the difference between the effect of factor A at high B vs. low B.
Interaction plot: non-parallel lines indicate a significant interaction.
Aliasing: in fractional/PB designs, interactions may be aliased with main effects. The alias structure is reported.

Summary Statistics

For each factor/level combination, the analysis computes:

Mean response at each level
Standard deviation at each level (when replicates exist)
Min and max response at each level
Sample count per level
Best level (highest mean for maximize, lowest for minimize)

Ordinal Trend Analysis

For factors with "type": "ordinal", additional trend analysis is performed:

Monotonic trend test: determines if the response increases or decreases consistently with the ordinal levels.
Trend direction: increasing, decreasing, or non-monotonic.
Trend strength: Spearman rank correlation between level order and response mean.

Knee Point Detection

When --knee is specified (or for sweep designs), the analysis identifies the point of diminishing returns:

Knee point location: the factor value where the response curve transitions from steep to flat.
Algorithm: maximum curvature method. Finds the point where the second derivative magnitude is largest.
Saturation percentage: what fraction of the total response range is achieved at the knee point.

Model Diagnostics

When a regression model is fitted (RSM designs, LHS), the following diagnostics are reported:

Metric	Description	Ideal
R²	Coefficient of determination. Proportion of variance explained by the model.	> 0.80
Adjusted R²	R² penalized for number of terms. Decreases if insignificant terms are added.	> 0.75
PRESS	Predicted Residual Error Sum of Squares. Leave-one-out cross-validation metric.	Small relative to SS_total
Predicted R²	1 - PRESS / SS_total. Measures predictive ability on held-out points.	> 0.70, within 0.2 of Adj R²
Leverage (h_ii)	Diagonal of the hat matrix H = X(X'X)^-1X'. Identifies influential runs.	< 2p/n (where p = parameters, n = runs)

Part 5: Output Formats

Result JSON

Each test script invocation must output a JSON file containing the measured response values. This is the contract between the test script and the analysis engine.

Result JSON (single run)

{
  "yield": 85.3,
  "purity": 99.1,
  "cost": 12.50
}

Keys must match the name fields in the responses array.
Values must be numeric (int or float).
The file is written to out_directory/run_NNN.json where NNN is the zero-padded run number.
If the test script fails, no result file should be written (the run is marked as failed).

Design Matrix JSON

The design_matrix.json file is generated alongside the runner script. It contains the complete design in a machine-readable format.

design_matrix.json structure

{
  "metadata": { "name": "...", "design_type": "full_factorial", "n_runs": 8 },
  "factors": ["temperature", "pressure", "catalyst"],
  "runs": [
    { "run": 1, "temperature": "80", "pressure": "2.0", "catalyst": "A" },
    { "run": 2, "temperature": "60", "pressure": "2.0", "catalyst": "B" },
    ...
  ]
}

CSV & TSV Export

The doe export-data command produces a flat file with one row per run.

CSV format

run,temperature,pressure,catalyst,yield,purity,cost
1,80,2.0,A,85.3,99.1,12.50
2,60,2.0,B,72.1,98.5,10.20
...

First column is always run (1-based run number in execution order).
Factor columns appear in definition order.
Response columns appear after factor columns, in definition order.
TSV format uses tab separators instead of commas.
With --partial, missing response values are left empty.

Worksheet Formats

The doe export-worksheet command produces a blank template for manual data recording.

CSV Worksheet

CSV worksheet

run,temperature,pressure,catalyst,yield,purity,cost
1,80,2.0,A,,,
2,60,2.0,B,,,
...

Markdown Worksheet

Markdown worksheet

| run | temperature | pressure | catalyst | yield | purity | cost |
|-----|-------------|----------|----------|-------|--------|------|
| 1   | 80          | 2.0      | A        |       |        |      |
| 2   | 60          | 2.0      | B        |       |        |      |
...

HTML Report

The analysis HTML report (doe report or doe analyze) is a self-contained file that includes:

Experiment summary: metadata, design type, factor/response definitions, run count.
Design matrix: full table of all runs and their factor settings.
ANOVA tables: one per response variable.
Effect estimates: main effects and interactions with confidence intervals.
Plots (embedded as base64 PNG): Pareto chart, main effects plots, interaction plots, normal/half-normal probability plots, residual diagnostics.
Optimization results: optimal factor settings for each response.
Design evaluation: D-efficiency, A-efficiency, G-efficiency metrics.

Part 6: Statistical Formulas

This section documents the key formulas used by the analysis engine. All computations follow standard DOE textbook conventions (Montgomery, Box-Hunter-Hunter).

Effect Calculation

Two-level factors

Effect = mean(Y_high) - mean(Y_low)

The main effect is the difference in average response between the high (+1) and low (-1) levels, averaged over all other factors.

Multi-level factors

Effect = max(Ȳ₁, Ȳ₂, ..., Ȳ_L) - min(Ȳ₁, Ȳ₂, ..., Ȳ_L)

For factors with more than 2 levels, the effect is the range of the level means — the maximum average response minus the minimum average response.

Interaction effect (two-level)

AB interaction = ½ [ (mean(Y_A+,B+) + mean(Y_A-,B-)) - (mean(Y_A+,B-) + mean(Y_A-,B+)) ]

ANOVA Decomposition

SS_total = ∑_i (Y_i - Ȳ)²

SS_factor = ∑_j=1^L n_j (Ȳ_j - Ȳ)²

MS_factor = SS_factor / DF_factor

F = MS_factor / MS_error

Where Ȳ_j is the mean response at level j, n_j is the count at level j, and Ȳ is the grand mean. The p-value is computed from the F-distribution with (DF_factor, DF_error) degrees of freedom.

Lenth's Pseudo Standard Error

s₀ = 1.5 × median(|c₁|, |c₂|, ..., |c_m|)

PSE = 1.5 × median(|c_i| : |c_i| < 2.5 × s₀)

Where c_i are the contrast (effect) estimates. Lenth's method is used for unreplicated 2-level factorial designs where there is no independent estimate of error. The first step computes a preliminary estimate s₀, then the PSE refines it by excluding effects larger than 2.5 × s₀ (likely active effects).

Confidence Intervals

CI = effect ± t_{α/2, df_error} × SE

SE = √(MS_error × (1/n₊ + 1/n_-))

For balanced 2-level designs: SE = √(4 × MS_error / N), where N is the total number of runs. The t-quantile uses the degrees of freedom from the error term.

D-efficiency

D-efficiency = 100 × (|X'X| / n)^1/p / n

Where X is the model matrix (n × p), n is the number of runs, and p is the number of model parameters. A D-efficiency of 100% means the design is D-optimal for the assumed model. See Part 7 for interpretation.

Desirability Functions

Individual desirability (maximize)

d_i = ((y_i - worst) / (best - worst))^s

Where worst and best come from the bounds field on the response. The exponent s controls the shape: s = 1 (linear), s > 1 (emphasis on reaching best), s < 1 (diminishing returns near best). Default s = 1. Values below worst get d = 0; values above best get d = 1.

Individual desirability (minimize)

d_i = ((worst - y_i) / (worst - best))^s

For minimize, worst > best. Values above worst get d = 0; values below best get d = 1.

Overall desirability

D = (∏ d_i^w_i)^{1 / ∑ w_i}

The weighted geometric mean of individual desirabilities. If any single d_i = 0, the overall D = 0 regardless of other responses. Weights w_i come from the weight field on each response.

Part 7: Design Evaluation Metrics

These metrics quantify how well a design supports parameter estimation. They are reported by doe info and included in analysis reports.

D-efficiency

D-efficiency = 100 × (|X'X|^1/p) / n

Interpretation: D-efficiency measures the overall precision of parameter estimates. It is proportional to the p-th root of the determinant of the information matrix X'X, normalized by the number of runs. Higher values mean smaller confidence ellipsoids for the estimated coefficients.

Range	Interpretation
90–100%	Excellent. Near-optimal for parameter estimation.
70–89%	Good. Suitable for most applications.
50–69%	Moderate. Consider augmenting the design.
< 50%	Poor. The design may not reliably estimate all parameters.

ℹ

Orthogonal designs (full factorial, Plackett-Burman) achieve 100% D-efficiency for the main-effects model. D-optimal designs maximize this metric by construction.

A-efficiency

A-efficiency = 100 × p / trace((X'X)^-1)

Interpretation: A-efficiency measures the average variance of parameter estimates. It is inversely proportional to the trace (sum of diagonal elements) of (X'X)^-1. Minimizing the trace minimizes the average variance across all coefficients.

A-efficiency focuses on average precision, while D-efficiency focuses on overall precision.
A design can have high D-efficiency but lower A-efficiency if some parameters are estimated much more precisely than others.
For orthogonal designs, A-efficiency = D-efficiency = 100%.

G-efficiency

G-efficiency = 100 × p / (n × max(h_ii))

Where h_ii are the diagonal elements of the hat matrix H = X(X'X)^-1X'.

Interpretation: G-efficiency measures the worst-case prediction variance across the design space. It is related to the maximum leverage point. A design with high G-efficiency has relatively uniform prediction variance — no single point dominates the model.

G-efficiency = 100% means all design points have equal leverage (balanced design).
Lower G-efficiency indicates that some regions of the factor space are predicted with much less precision than others.
Particularly important for response surface designs where prediction across the entire space matters.

⚠

No Single Metric Is Sufficient

Always examine D-, A-, and G-efficiency together. A design optimized for one criterion may perform poorly on another. For most practical purposes, D-efficiency above 70% with G-efficiency above 50% indicates an adequate design.

← User Guide Theory →

Technical Reference

Part 1: CLI Reference

Global Behavior

doe generate

doe analyze

doe info

doe optimize

doe report

doe record

doe status

doe power

doe augment

doe init

doe export-worksheet

doe export-data

doe next-batch

doe suggest

doe scaffold-config

doe scaffold-test

doe simulate

doe compare

doe trend

doe sensitivity

doe calibrate

doe archive

doe serve

Part 2: Configuration Schema

Top-level Fields

Factor Object

Level Values Are Strings

ResponseVar Object

RunnerConfig

Settings

Part 3: Design Types

full_factorial

Full Factorial Design

fractional_factorial

Fractional Factorial Design

plackett_burman

Plackett-Burman Design

latin_hypercube

Latin Hypercube Sampling

central_composite

Central Composite Design (CCD)

box_behnken

Box-Behnken Design

definitive_screening

Definitive Screening Design (DSD)

taguchi

Taguchi Orthogonal Array

d_optimal

D-Optimal Design

mixture_simplex_lattice

Mixture Simplex-Lattice Design

mixture_simplex_centroid

Mixture Simplex-Centroid Design

linear_sweep

Linear Parameter Sweep

log_sweep

Logarithmic Parameter Sweep

split_plot

Split-Plot Design

Choosing a Design

Part 4: Analysis Output

ANOVA Table

Error Estimation Methods

Effect Estimates

Interaction Effects

Summary Statistics

Ordinal Trend Analysis

Knee Point Detection

Model Diagnostics

Part 5: Output Formats

Result JSON

Design Matrix JSON

CSV & TSV Export

Worksheet Formats

CSV Worksheet

Markdown Worksheet

HTML Report

`doe generate`

`doe analyze`

`doe info`

`doe optimize`

`doe report`

`doe record`

`doe status`

`doe power`

`doe augment`

`doe init`

`doe export-worksheet`

`doe export-data`

`doe next-batch`

`doe suggest`

`doe scaffold-config`

`doe scaffold-test`

`doe simulate`

`doe compare`

`doe trend`

`doe sensitivity`

`doe calibrate`

`doe archive`

`doe serve`