Skip to content

StudyManifest v1 acceptance specification

Status: normative persisted-state specification; SA-001..047, SA-049..055, SA-061..073, and Python/CLI lifecycle operations are executable under the bounded-slice clarifications below; SA-048 remains an explicit future environment-override case and SA-074 completes with caller migration

Contract: Durable study and StudyManifest v1

ADR: ADR 0008

These 74 scenarios are the complete v1 acceptance inventory. Each row freezes the observable contract before production implementation. “Files” names the canonical files that must exist or remain unchanged. “Python” names the result or typed error. “CLI JSON” is the stable vamos.study-command-result/1 outcome; human text must carry the same state, reason, and next action. Exit codes apply to the CLI form. No scenario may be implemented as skipped or xfail.

The first production slice implements SA-001 through SA-020 and Python create/load. The second implements the bounded sequential meanings of SA-021 through SA-026 and SA-061 through SA-065. The third implements persisted fail-fast/continue, task-versus-infrastructure outcomes, graceful cancellation, forced-death nonpublication, and no-skip task selection from SA-027 through SA-032 plus SA-068 in tests/experiment/study_manifest/. Stale-attempt reconciliation from SA-031 and the recovery cases SA-033..047 and SA-049..055 are implemented by explicit single-process operations. SA-048's environment-change acceptance option, coordination, parallelism, and caller migration remain roadmap requirements rather than available behavior. The single-owner CLI and derived summary output cover the applicable SA-069 through SA-073 command behavior.

For this bounded recovery slice, SA-031 and SA-036 mean immediate explicit single-owner reconciliation after a prior process has ended; there is no lease or lease-expiry protocol. SA-038 covers only detectable same-process ownership and byte-stable refusal. Persistent locks, live cross-process ownership, leases, heartbeats, and fencing remain Goal 6 work.

Read-only planning has its own contiguous PL-001 through PL-021 acceptance specification and does not alter or renumber this persisted-state inventory.

Module paths describe intended ownership; implemented rows use their current production owner. For direct library-only validation, deferred “CLI JSON” and “Exit” cells specify a future command boundary, not an available command.

Specification, identity, and creation

ID Proposed module/function Precondition Input Operation Expected canonical files Expected states Python result CLI JSON Exit Invariant Defect prevented
SA-001 study.models.StudySpec.validate No study exists Minimal supported matrix and defaults Validate spec No files No state Valid immutable StudySpec ok=true, operation create 0 Default policy is fail_fast and max attempts is 3 Hidden or mutable defaults
SA-002 study.models.StudySpec.validate No study exists Explicit empty matrix Validate and create Manifest, spec, plan, event 1; no task dirs Study created, zero tasks Study with zero-count report state=created, task_count=0 0 Empty planned studies are representable Empty-set crash or fabricated task
SA-003 study.planning.resolve_spec Supported built-ins are available Same matrix in two display orders Resolve both No files before create No state Equal task set and plan_id Equal reported plan_id 0 Plan identity is set based Order-dependent scientific identity
SA-004 study.planning.compute_task_id Complete run resolution succeeds Two resolved specs differing only in seed Compute identities No files No state Distinct RunManifest-compatible task IDs Distinct task IDs 0 Every scientific resolved field participates Task collision across seeds
SA-005 study.planning.compute_plan_id Task projections are valid Same task set, shuffled Compute plan IDs No files No state Equal sha256: IDs Equal plan_id 0 Sorted task projections define plan identity Matrix-order drift
SA-006 study.planning.compute_plan_id A plan resolves Change labels, metadata, output, worker count Recompute No files No state Unchanged task and plan IDs Unchanged IDs 0 Presentation and execution placement are excluded Labels changing science identity
SA-007 study.planning.compute_task_id A plan resolves Change a scientific algorithm or budget field Recompute No files No state Changed task and plan IDs Changed IDs 0 Scientific changes create a new plan Silent configuration reuse
SA-008 study.validation.verify_plan Published study is valid Modified, appended, or deleted plan task Load Existing tree unchanged Stored state unchanged PlanMismatchError reason=PLAN_MISMATCH 4 Published plan is immutable In-place study mutation
SA-009 study.serialization.canonical_json JSON-compatible document exists Same object with varied key insertion Serialize No files No state Byte-identical sorted compact UTF-8 Stable hashes 0 Canonical JSON has one byte form Platform-dependent hashes
SA-010 study.serialization.load_json A path is confined Object repeats a key Load Files unchanged State not trusted MalformedStudyError reason=DUPLICATE_JSON_KEY 3 Duplicate keys are rejected Parser-dependent meaning
SA-011 study.serialization.decode_document Supported identity/version Document contains an unknown field Decode Files unchanged State not trusted MalformedStudyError reason=UNKNOWN_FIELD 3 V1 schemas are closed Misspelled or smuggled fields
SA-012 study.serialization.canonical_json Document contains a number NaN, positive infinity, or negative infinity Encode or load No new files State not trusted MalformedStudyError reason=NON_FINITE_NUMBER 3 Canonical documents contain finite JSON Nonportable numeric state
SA-013 study.serialization.decode_document Document identity is known Future or malformed schema version Load Files unchanged State not trusted UnsupportedStudySchemaError reason=UNSUPPORTED_SCHEMA 4 Readers never guess future semantics Accidental forward compatibility
SA-014 study.identity.validate_uuid4 A document is decoded Uppercase, non-v4, or malformed UUID Validate Files unchanged State not trusted MalformedStudyError reason=INVALID_IDENTITY 3 Study, execution, attempt, event, and run IDs are lowercase UUIDv4 Ambiguous entity identity
SA-015 study.paths.confined_path Study root is known Absolute, drive, URI, backslash, empty, traversal, NUL, or symlink escape Resolve reference Files unchanged State not trusted UnsafeStudyPathError reason=UNSAFE_PATH 3 Load targets stay under the root Path traversal and relocation breakage
SA-016 study.creation.create_study Destination already exists in any form Valid spec and existing empty or partial path Create Existing destination byte-identical No new state StudyOutputCollisionError with execution false reason=OUTPUT_COLLISION, execution_occurred=false 5 No overwrite or force path exists Destructive creation
SA-017 study.creation.publish_study Absent destination and injectable crash boundary Crash before terminal directory rename Create No destination; only owned staging may remain for cleanup No published state StudyInfrastructureError reason=ATOMIC_PUBLICATION_FAILED 7 A visible study is complete or absent Half-created canonical tree
SA-018 study.loading.load_study Valid complete study is copied as a whole New root path Load and inspect copy All relative descriptors still match Same persisted state Verified StudyReport ok=true, unchanged IDs 0 Identity and references are location independent Absolute-path persistence
SA-019 study.creation.create_study Absent destination and nonempty resolved plan Valid spec Create without run Manifest, spec, plan, event 1, one task file per digest Study created; every task pending; no attempts Study and StudyReport(changed=true) state=created, next action run 0 Creation performs zero optimization Create-and-run coupling
SA-020 study.loading.load_study Valid planned study exists Root path Data-only load Tree byte-identical State created and tasks pending Persisted Study handle changed=false 0 Loading has no execution or mutation Reads with side effects

Sequential execution and persisted failure policy

ID Proposed module/function Precondition Input Operation Expected canonical files Expected states Python result CLI JSON Exit Invariant Defect prevented
SA-021 Study.run / study.execution.execute_study Pristine study created; no same-process reentry No arguments; persisted plan Start sequential run execution_started precedes claim/start events and checkpoints Study/task/attempt are durably running before objective evaluation Fresh immutable Study after terminal completion; running state is data-only observable during work Deferred Future 0 Execution start commits before optimization Invisible execution start
SA-022 study.execution._run_pending_task Next canonical task pending Fresh execution ID; no worker/lease Reserve sequential attempt Created attempt checkpoint, task_claimed, attempt_started, task/root checkpoints; no coordination file Task and attempt running; claim epoch incremented Final returned Study exposes attempt identity Deferred Future 0 Sequential reservation creates one UUIDv4 attempt and no lease fiction Duplicate or fabricated claim
SA-023 study.execution._execute_and_commit Matching durable task/attempt are running Persisted resolved task Begin objective evaluation Running event and checkpoints already durable Attempt running before first evaluation Objective can data-only load the running attempt Deferred Future 0 User work never begins before durable running state Crash recorded as never started
SA-024 study.run_publication.publish_success Running attempt; canonical result exists Persisted task identity and result Publish and verify success Canonical run first; full verify/reload; success event; terminal attempt/task/root checkpoints Attempt/task succeeded; study remains running while pending tasks exist Fresh Study links only the verified run Deferred Future 0 No success precedes verified canonical run Manifest claiming missing output
SA-025 study.execution._complete_running Every task succeeded Execution ID Finalize study_completed event and root checkpoint Study completed Fresh Study(state=completed); old handle unchanged Deferred Future 0 Completed means all tasks succeeded Partial work reported complete
SA-026 study.execution.execute_study Sequential study has several pending tasks Immutable sorted plan Run one task at a time Claim sequence and runs in ascending task_id order Tasks finish in task_id order; plan_index remains presentation metadata Returned handle retains both fields without conflating them Deferred Future 0 Selection ignores filesystem, mapping, timestamp, and display order Nondeterministic baseline behavior
SA-027 study.execution.handle_task_failure Policy fail_fast; one task fails Sanitized execution failure Commit failure Failed run when publishable, failure event/checkpoints, pause event; later tasks untouched Attempt/task failed; study paused; unclaimed tasks pending Partial StudyReport with next actions state=paused, failed ID, resume/retry commands 6 Fail-fast stops new claims only after durable failure Lost failure or skipped pending work
SA-028 study.execution.handle_task_failure Policy continue; an independent task fails Sanitized execution failure Continue remaining tasks and finalize Failure plus later attempt/run files; completion-with-failures event Failed task retained; other tasks terminal; study completed_with_failures Complete partial StudyReport state=completed_with_failures and counts 6 Continue retains all outcomes and claims all independent pending tasks First-error truncation
SA-029 study.execution.fail_infrastructure Canonical state remains writable Journal, storage, scheduler, or verifier failure Record infrastructure failure study_failed event and root checkpoint; no fabricated task result Study failed; affected task unchanged or reconciled StudyInfrastructureError and trustworthy report reason=STUDY_INFRASTRUCTURE_FAILURE 7 Infrastructure failure is not a task failure Misclassified scientific result
SA-030 study.execution.cancel Created, running, or paused nonterminal study User cancellation Stop claims and settle active work Cancellation attempt/task events and final study event Active attempts terminal; unclaimed tasks cancelled; study cancelled StudyReport(state=cancelled) state=cancelled, next action inspect 8 Cancellation accounts for every task Orphaned pending/running state
SA-031 study.recovery.reconcile Process died during running attempt Active then expired lease Load then recover No post-death write; later interruption event/checkpoints after expiry Active lease stays running; stale unresolved attempt becomes interrupted and study paused Verified recovery report state=paused, reason=INTERRUPTED 8 Process death never invents failure or success False terminal outcome
SA-032 study.execution.skip_task Applicability rule explicitly rejects a pending task under continue policy Stable bounded reason Skip task_skipped event and task/root checkpoints Task skipped; final study completed_with_failures when no runnable task remains Report exposes reason Skipped count and reason 6 Only explicit applicability may skip Fail-fast mislabeled as skip

Journal recovery, resume, and retry

ID Proposed module/function Precondition Input Operation Expected canonical files Expected states Python result CLI JSON Exit Invariant Defect prevented
SA-033 study.journal.derive_effective_study Valid event exists beyond an older root/task checkpoint Study root Data-only load Every canonical byte unchanged; no repair write Loaded effective state equals replay while stored checkpoints may lag Immutable Study at effective event head Deferred Future 0 Valid journal events outrank stale checkpoints without making load executable Lost committed transition or read-side repair
SA-034 study.journal.verify_chain Event filenames contain a gap or duplicate sequence Study root Load No files changed State not trusted StudyIntegrityError reason=EVENT_HASH_CHAIN_BROKEN 3 Event sequence is contiguous and unique Partial history accepted
SA-035 study.journal.verify_chain Event previous_event_sha256 does not match predecessor Study root Load No files changed State not trusted MalformedStudyError reason=EVENT_HASH_CHAIN_BROKEN 3 Every event is chained to the previous canonical bytes/hash Rewritten audit history
SA-036 study.recovery.reconcile_attempt Running attempt lease expired; expected run absent/incomplete Study root Reconcile Interruption and lease-reclaimed events; checkpoints; stale lease removed Attempt/task interrupted; study paused Recovery report with retryability reason=STALE_ATTEMPT_INTERRUPTED 8 Stale unresolved work is interrupted exactly once Infinite running state
SA-037 study.recovery.reconcile_attempt Run was atomically published before process died; success event absent Complete matching verified run Reconcile One recovered success event and checkpoints Attempt/task succeeded; study derived accordingly Recovery report references existing run changed=true, recovered success 0 Complete output is verified before a new attempt Duplicate successful execution
SA-038 study.recovery.reconcile_attempt Running attempt has unexpired matching lease Resume request Verify Files unchanged Attempt/task remain running; study remains running StudyLockedError or active-work report reason=ACTIVE_LEASE 5 Live ownership is not reclaimed Concurrent double execution
SA-039 study.resume.resume Study has selected successful tasks and pending work Resume request Reconcile and run No new attempt/run for successful tasks; new records only for eligible tasks Success unchanged; pending tasks advance StudyReport Counts show successes retained 0 or 6 Resume never reruns success Duplicate canonical runs
SA-040 study.validation.verify_selected_run Succeeded task references a missing run file/tree Resume request Verify Files unchanged Stored state reported corrupt, not pending ReferencedRunMissingError reason=REFERENCED_RUN_MISSING 3 Missing success is an integrity error Silent replacement of lost evidence
SA-041 study.validation.verify_selected_run Referenced run bytes, ID, task ID, or semantic hash differs Resume request Verify Files unchanged Stored state reported corrupt ReferencedRunCorruptError Exact hash or identity mismatch reason 3 Every success reference is fully verified Tampered or wrong run accepted
SA-042 study.validation.verify_plan Persisted spec/plan descriptor or plan ID changed Resume request Verify Files unchanged Stored state unchanged and operation rejected PlanMismatchError reason=PLAN_MISMATCH 4 Resume uses the immutable published plan Current defaults changing old work
SA-043 study.resume.resume Paused study contains pending tasks Default resume request Start new execution New execution event and attempts only for pending eligible tasks Study running, then derived terminal/paused state StudyReport with parent execution ID operation=resume, eligible count 0 or 6 Pending is the default resume set Resume doing nothing after fail-fast
SA-044 study.resume.resume Paused study contains reconciled interrupted task below limit retry_interrupted=true default-supported request Resume New attempt ID/number, execution and run ID; old attempt immutable Interrupted task returns to running, then terminal StudyReport Attempt lineage shown 0 or 6 Interrupted work gets a new attempt Reopening terminal attempt records
SA-045 study.resume.resume Study contains failed tasks Request without then with explicit failed selection Resume twice as applicable First call creates no failed-task attempt; explicit call creates one Failed unchanged first; then running Unchanged report then execution report changed=false then selected retry 6 then 0 or 6 Failure retry requires explicit consent Automatic retry
SA-046 study.resume.resume Verified study has no eligible runnable task Resume request Verify and derive Tree byte-identical State unchanged StudyReport(changed=false) with next action changed=false; state-derived reason 0 for completed, 6 for valid partial, 4 for invalid operation No-op resume is data-stable Manufactured transitions
SA-047 study.environment.verify_resume Persisted and current material environments differ Resume request Verify before claim Files unchanged State unchanged ResumeEnvironmentIncompatibilityError reason=RESUME_ENVIRONMENT_INCOMPATIBLE 7 Default resume refuses material environment drift Incomparable continuation
SA-048 study.environment.accept_change A future Goal exposes explicit acceptance; components/spec fully validate accept_environment_change=true Resume Environment-change event records old/new fingerprints before claim Study may enter running without identity change StudyReport environment_change_accepted=true 0 or 6 Explicit acceptance is audited and never changes task/plan ID Unrecorded environment override

Retry and local concurrency

ID Proposed module/function Precondition Input Operation Expected canonical files Expected states Python result CLI JSON Exit Invariant Defect prevented
SA-049 study.retry.select_tasks Failed or interrupted retryable tasks exist Ordinary run/resume without retry selection Execute No retry attempts for failed tasks Failed tasks unchanged Report identifies explicit retry command changed=false or pending-only result 6 V1 has no implicit retry Hidden extra compute
SA-050 study.retry.retry Failed retryable task is below attempt limit failed_only=true Claim retry New attempt/event/lease and later new run; old records unchanged Task running, then terminal; study re-enters running StudyReport Selected failed task and attempt number 0 or 6 Retry preserves task ID and history Destructive attempt replacement
SA-051 study.retry.retry Interrupted retryable task is below limit Explicit interrupted selection Claim retry New attempt and execution/run identities Task running, then terminal StudyReport Attempt lineage and outcome 0 or 6 Interruption retry is an explicit new attempt Resuming an old process identity
SA-052 study.retry.enforce_limit Task already has max_attempts_per_task attempts Retry selection Refuse before claim Tree byte-identical State unchanged RetryLimitError reason=RETRY_LIMIT_REACHED 4 Persisted attempt bound is absolute Unbounded retry loop
SA-053 study.retry.classify Failure category is nonretryable Retry selection Refuse Tree byte-identical State unchanged RetryNotAllowedError reason=NONRETRYABLE_FAILURE 4 Spec, integrity, deterministic config, transition, and exact mismatch failures cannot retry Repeated deterministic failure
SA-054 study.retry.retry Task is succeeded, skipped, or cancelled Any force-like retry request Refuse Tree byte-identical Terminal state unchanged RetryNotAllowedError reason=TERMINAL_TASK_NOT_RETRYABLE 4 V1 exposes no successful force retry Multiple authoritative successes
SA-055 study.retry.create_attempt Failed/interrupted task is eligible Retry claim Allocate identities New attempt record and expected run path Attempt created with next number Distinct attempt, execution, and run UUIDs; same task ID All lineage IDs 0 Entity identities are never conflated Run/attempt aliasing
SA-056 study.coordination.acquire_study_lock No live lock exists Worker UUID and random token Acquire for metadata commit Operational lock only; canonical state unchanged Canonical states unchanged StudyLock expiring in 30 seconds Lock acquisition is not canonical output 0 Optimization never holds the short study lock Global serialization of compute
SA-057 study.coordination.claim_task Another worker owns an unexpired task lease Same task Claim Files unchanged Task/attempt remain owned/running TaskAlreadyClaimedError reason=ACTIVE_LEASE 5 Active leases cannot be stolen Double execution
SA-058 study.coordination.publish_attempt Worker has stale epoch or wrong lease token Run outcome Publish Canonical state and selected run unchanged Current owner state unchanged LeaseLostError reason=LEASE_FENCE_REJECTED 5 Task ID, attempt ID, epoch, and token compare-and-set Zombie worker publication
SA-059 study.coordination.reclaim_lease Lease expired Expected run path and new worker Reconcile then reclaim Recovered outcome or interruption/reclaim events before new claim Old attempt terminal before new attempt Reconciliation report or TaskClaim Recovery and new epoch reported 0 or 8 Reconcile always precedes epoch increment and reclaim Overwriting late valid output
SA-060 study.journal.commit_event Two local workers complete concurrently Two owned outcomes Commit under study lock Two contiguous one-file events with valid hash order Both outcomes reflected; order equals durable commit order Reports converge on same event head Final counts and head agree 0 or 6 Audit order is total even when finish order varies Split-brain checkpoints

Run ownership, security, API, and caller integration

ID Proposed module/function Precondition Input Operation Expected canonical files Expected states Python result CLI JSON Exit Invariant Defect prevented
SA-061 study.run_reference.validate Plan task and canonical run exist RunManifest with task identity Attach run Attempt holds bounded descriptor only Outcome transition allowed only on exact task match Valid RunReference Matching run/task IDs 0 Study task ID exactly equals RunManifest task ID Wrong run linked to task
SA-062 study.run_reference.from_manifest Verified canonical manifest exists Root-relative manifest path Build descriptor Attempt stores path, bytes, file SHA-256, semantic manifest hash, role and operation State unchanged until outcome event Immutable descriptor Descriptor fields in detailed output 0 Both byte and semantic integrity are recorded Hash ambiguity
SA-063 study.models.AttemptRecord.validate Attempt references a run Record containing copied spec, arrays, environment, provenance, or replay evidence Decode Files unchanged State not trusted MalformedStudyError reason=UNKNOWN_FIELD or ownership violation 3 RunManifest/run files solely own per-run truth Divergent duplicate artifacts
SA-064 study.run_publication.commit_task_failure Reconstruction or objective fails after durable start Sanitized in-process failure Commit policy-owned task failure Verified failed run first when publishable, failure event second, terminal attempt/task/root checkpoints, then policy terminal event where applicable Attempt/task failed; study paused for fail-fast or remains running for continue; later tasks follow persisted policy Fresh partial Study after policy finalization Deferred Future 6 Persisted policy applies only after durable task evidence Exception-only state or competing fixed stop
SA-065 study.run_publication.publish_success Execution cannot publish or verify a terminal success Interrupted publication or verification Stop without reconciliation No terminal outcome event or run reference; any complete orphan run remains unreferenced Attempt/task/study remain explicitly running; no success/failure is fabricated StudyRunPublicationError or StudyRunVerificationError Deferred Future 7/8 Absence of verified linked output cannot become an outcome; reconciliation is deferred False success or false failure artifact
SA-066 study.paths.confined_path Malicious root includes external symlink or unsafe reference Load, inspect, summarize, or resume Resolve safely Outside target never read or written; tree unchanged State not trusted UnsafeStudyPathError reason=UNSAFE_PATH 3 Every operational path is confined without following symlinks Arbitrary filesystem access
SA-067 study.loading.load_study Documents contain plugin-like strings and unavailable components Load or inspect only Parse and verify data Tree byte-identical; no imports, shell, network, registry resolution, pickle, or optimization Persisted state reported when structurally valid Data-only Study or report changed=false 0 Reading data never executes it Manifest code execution
SA-068 study.errors.sanitize_failure Underlying exception contains token, username, host, environment, traceback, or absolute path Persist failure Sanitize and bound Failure record/event contain stable safe fields only Correct failure state Typed error with sanitized message and safe action Same bounded fields, no secrets State-derived Canonical data never leaks uncontrolled diagnostics Secret and personal-path disclosure
SA-069 vamos.create_study and vamos.load_study Goal 1 implementation is installed Valid StudySpec and output, then root path Create then data-only load Planned canonical tree only Study created; tasks pending Public Study handles with equal IDs study create equivalent reports IDs and run command 0 Top-level factories are the supported entry points Internal-module dependency
SA-070 study.execution.execute_study and study.recovery Public Study handle exists run, resume, or retry call Delegate One common journal/claim/run/checkpoint protocol Operation-specific valid transitions StudyReport from one service family Same command-result schema State-derived No second retry or CLI execution path Behavioral drift among entry points
SA-071 experiment.cli.study Any valid command result or typed error --json Emit Canonical study files only when operation requires Reported state equals verified state Same library result/error One UTF-8 vamos.study-command-result/1 object on stdout; progress only on requested stderr 0–8 by mapping JSON mode has no prompt, ANSI, or mixed stdout Unparseable automation output
SA-072 experiment.cli.study.map_exit_code A typed outcome exists Success, usage, corruption, transition, conflict, task partial, infrastructure, or interruption Map No additional files State unchanged by mapping Original result/error retained Exit respectively 0, 2, 3, 4, 5, 6, 7, or 8 Exact mapping One category never changes exit family Scripts confusing partial and corrupt states
SA-073 study.summary.summarize Valid study and referenced runs exist Optional derived destination Generate, delete, regenerate Only derived/ or explicit output changes; summary names study/plan/time/root hash/event head Canonical states unchanged StudySummary with explicit every-task outcome operation=summarize, changed=false for canonical state 0 Derived files are deletable and never drive resume Summary becoming authority
SA-074 canonical caller surface Canonical surface covers callers Ablation, benchmark, Studio, analysis, and experiment runner workflows Delegate and remove the superseded layer One StudyManifest layout plus canonical run dirs States come only from the canonical study reader Public study reports; top-level multi-seed StudyResult remains distinct One vamos study group; callers expose canonical IDs 0 or state-derived No second runtime, CSV authority, or manifest-directory heuristic remains Dual authorities and compatibility debt

Coverage rule

The contract examples cover representative rows, while the implementation roadmap makes every row executable in its owning Goal. Repository validation must assert that the IDs are exactly the contiguous set SA-001 through SA-074, that all twelve fields are populated, and that no duplicate acceptance ID or undocumented outcome is introduced.