Skip to content

CLI and config files

The standard single-run CLI path, canonical run commands, and the single-owner durable-study lifecycle are covered by command-level smoke tests. Heavier tuning and broader benchmark matrices still depend on installed extras and the local environment.

Quickstart wizard

Run an interactive wizard that writes a config file and executes a single run:

vamos quickstart

If you are new to Python, start with docs/guide/minimal-python.md.

List available templates:

vamos quickstart --template list

Run a template non-interactively:

vamos quickstart --template physics_design --yes --no-plot

Skip optional dependency warnings:

vamos quickstart --no-preflight

Template keys (short list):

  • demo: quick benchmark demo (ZDT1)
  • physics_design: mixed-variable structural design (welded beam)
  • bio_feature_selection: real-data feature selection (requires examples extra)
  • chem_hyperparam_tuning: SVM hyperparameter tuning (requires examples extra)

The config is saved under results/quickstart/ and can be re-run with vamos --config <path>.

Results helpers

Summarize recent runs:

vamos summarize --results results

Show only the latest run:

vamos summarize --latest

Open the latest run folder:

vamos open-results --open

Inspect one canonical run without materializing arrays:

vamos results inspect results/ZDT1/nsgaii/numpy/seed_7
vamos results inspect results/ZDT1/nsgaii/numpy/seed_7 --json

Fully verify integrity and exact replay compatibility without optimization:

vamos results verify results/ZDT1/nsgaii/numpy/seed_7
vamos results verify results/ZDT1/nsgaii/numpy/seed_7 --require-level exact

Execute a verified same-environment built-in replay as a new canonical run:

vamos reproduce results/ZDT1/nsgaii/numpy/seed_7
vamos reproduce results/ZDT1/nsgaii/numpy/seed_7 --output results/replays/zdt1-seed-7

Add --json to any of these commands for one machine-readable stdout document. Replay never overwrites or modifies its source.

Durable studies

Resolve a durable study before creating a directory or executing an objective:

{
  "problems": ["zdt1", "zdt2"],
  "algorithms": ["nsgaii"],
  "seeds": [0, 1],
  "max_evaluations": 10000,
  "pop_size": 80
}

vamos study plan study.json
vamos study plan study.json --output studies/comparison-01
vamos study plan study.json --json
vamos study create study.json --output studies/comparison-01
vamos study run studies/comparison-01
vamos study inspect studies/comparison-01 --json
vamos study summarize studies/comparison-01
vamos study summarize studies/comparison-01 --format csv --output artifacts/studies/tasks.csv

Planning is read-only: it creates no study, runs no task, and does not reserve the proposed output. Its plan_id and task IDs match later Python vamos.create_study(...) creation from the same StudySpec. Every study command emits exactly one vamos.study-command-result version 1.0.0 document in JSON mode. Creation and execution are separate. inspect and an in-memory summarize are read-only; JSON/CSV summary files are written only when --output is explicit and never overwrite an existing path.

Use vamos study resume STUDY_DIR for eligible pending/interrupted work and vamos study retry STUDY_DIR --failed for explicit bounded failed-task retry. Concurrent mutation is unsupported: one process must remain the only mutation owner for a study. There is no cross-process cancel command; foreground Ctrl+C uses graceful durable cancellation.

Main runner

Use vamos for single runs and problem sets.

Quick walkthroughs

Single run (default output under results/):

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 5000 --population-size 80 --seed 7

Python equivalent (preferred for scripting):

from vamos import optimize

result = optimize("zdt1", algorithm="nsgaii", max_evaluations=5000, pop_size=80, seed=7)

Run a predefined problem set with both internal algorithms:

vamos --problem-set families --algorithm both --max-evaluations 3000

Compare backends on one problem:

vamos --problem zdt1 --experiment backends --max-evaluations 2000

Optional backends need extras: numba and moocore require pip install -e ".[compute]" (or pip install "vamos-optimization[compute]"). Missing backends are skipped.

Multiprocessing evaluation for expensive problems:

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 8000 --eval-strategy multiprocessing --n-workers 4

Enable live visualization and save plots:

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 2000 --live-viz --plot

Early stop when hypervolume reaches a target fraction:

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 15000 --hv-threshold 0.9

Include external baselines (ZDT1 only):

vamos --problem zdt1 --algorithm both --include-external --external-problem-source native

Walkthrough: run and inspect outputs

1) Run a single optimization:

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 5000 --population-size 80 --seed 7

2) Inspect the canonical artifact under results/ (default):

  • manifest.json: requested/resolved configuration, actual seed, outcome, provenance, and hashes
  • result.npz: objective, decision, constraint, population, and archive arrays
  • environment.json: bounded runtime environment details

3) Save plots as presentation output outside the canonical run leaf:

vamos --problem zdt1 --algorithm nsgaii --max-evaluations 5000 --population-size 80 --seed 7 --plot

Key flags

  • --algorithm: nsgaii, moead, smsemoa, nsgaiii, spea2, ibea, smpso, both, or external baselines (pymoo_nsga2, jmetalpy_nsga2, pygmo_nsga2)
  • --engine: numpy | numba | moocore | auto. The deterministic default is numpy; use auto when you want heuristic backend selection.
  • --problem: any registry key (see Problems page)
  • --problem-set: predefined sets (e.g., families)
  • --validate-config: validate --config and exit
  • --output-root: directory for run artifacts (default: results/)
  • --no-preflight: skip optional dependency warnings
  • --population-size, --offspring-population-size
  • --max-evaluations
  • --hv-threshold and --hv-reference-front
  • --selection-pressure, --external-archive-size
  • --eval-strategy: serial | multiprocessing (with --n-workers)
  • --live-viz with --live-viz-interval, --live-viz-max-points
  • --plot: save Pareto front plots after runs
  • Variation overrides per algorithm (examples):
  • --nsgaii-crossover sbx --nsgaii-crossover-prob 1.0 --nsgaii-mutation pm --nsgaii-mutation-prob 1/n
  • --moead-crossover sbx --moead-mutation pm --moead-aggregation pbi
  • --smsemoa-mutation pm --nsga3-crossover sbx

Config files (YAML/JSON)

Use --config path/to/spec.yaml; CLI flags override file values.

version: "1"
defaults:
  title: My run
  algorithm: moead
  engine: numpy
  population_size: 120
  max_evaluations: 20000
  hv_threshold: 0.8
  moead:
    crossover: {method: sbx, prob: 1.0, eta: 20}
    mutation: {method: pm, prob: "1/n", eta: 20}
problems:
  bin_knapsack:
    algorithm: nsgaii
    n_var: 30
    population_size: 150
    nsgaii:
      crossover: {method: uniform}
      mutation: {method: bitflip, prob: "1/n"}

Validate a config without running:

vamos --config configs/experiment.yaml --validate-config

Run a config with a CLI override:

vamos --config configs/experiment.yaml --algorithm smsemoa --max-evaluations 10000

Other subcommands

All tools are accessed via vamos <subcommand>. Run vamos help for the full list.

  • Self-check: vamos check
  • Benchmarking: vamos bench --list and vamos bench ZDT_small --algorithms nsgaii moead --output report/
  • Fast benchmark verification: vamos bench ZDT_small --algorithms nsgaii --output report/ --smoke
  • Tuning: vamos tune --instances zdt1,zdt2,zdt3 --algorithm nsgaii --backend optuna --backend-fallback random --split-strategy suite_stratified --budget 5000 --tune-budget 200 --n-jobs -1
  • Ablation plans: vamos ablation --config configs/ablation.yaml
  • Profiling: vamos profile --problem zdt1 --engines numpy,numba --budget 2000 --output report/profile.csv
  • Problem zoo: vamos zoo list, vamos zoo info zdt1, vamos zoo run zdt1 --algorithm nsgaii --budget 3000
  • Studio (interactive, needs studio extra): vamos studio --study-dir results

Tuning quick notes (vamos tune)

Use this guide for quick usage. For the complete, maintained tune reference (all backends, split/fallback behavior, finisher/validation/test, and artifact contracts), see:

  • docs/topics/tuning.md

Recommended robust invocation:

vamos tune \
  --instances zdt1,zdt2,zdt3,dtlz1,dtlz2,wfg1 \
  --algorithm nsgaii \
  --backend optuna \
  --backend-fallback random \
  --split-strategy suite_stratified \
  --budget 5000 \
  --tune-budget 200 \
  --n-jobs -1

Quick verification path (built-in backend, tiny budgets):

vamos tune --instances zdt1,zdt2,zdt3,dtlz1,dtlz2,wfg1 --algorithm nsgaii --backend random --smoke --output-dir results/tuning_smoke

Ablation config example

algorithm: nsgaii
engine: numpy
output_root: results/ablation_demo
default_max_evals: 2000
problems: [zdt1]
seeds: [1, 2, 3]
base_config:
  population_size: 60
  offspring_population_size: 60
variants:
  - name: baseline
summary_dir: results/ablation_demo/summary

The CLI writes a summary CSV by default to <output_root>/summary/ablation_metrics.csv (override with summary_path or summary_dir).