Understand setup workflows and tools

When you start with /onyx ..., the orchestrator creates the repo-side setup files it needs to run comparable parallel research. You can review the setup before Research starts; changing setup during Research should stop the session, revise setup, and run validation again.

`onyx/setup.json`: Setup Policy

setup.json is the canonical local setup policy for the orchestrator and hypothesis workers. It defines what agents optimize, how measurement runs, what files are in scope, which setup paths are protected, what resources are leased, which tools are available, and the linear workflow for one experiment attempt. The orchestrator creates the first version and onyx setup validate writes the latest check results to onyx/validation.json. During Research, hypothesis workers treat setup files as frozen. If setup is wrong, stop the session and revise it. Example:

{
  "schemaVersion": 2,
  "goal": "Minimize tracking error for the arm controller without increasing overshoot.",
  "experimentPolicy": {
    "mode": "single_candidate",
    "maxDiagnosticSeconds": 30
  },
  "projectPath": "",
  "scope": {
    "editable": ["src/control"],
    "protected": [
      "onyx/setup.json",
      "onyx/validation.json",
      "onyx/onyx.md",
      "onyx/tools/"
    ]
  },
  "metric": {
    "name": "tracking_error",
    "unit": "error",
    "direction": "minimize"
  },
  "resources": {
    "simulator": { "slots": 1, "description": "Controller simulator" }
  },
  "tools": {
    "reset.clean": {
      "command": "bash",
      "args": ["onyx/tools/reset.sh"],
      "resources": ["simulator"],
      "timeoutSeconds": 60
    },
    "evaluation.run": {
      "command": "bash",
      "args": ["onyx/tools/evaluation/run.sh"],
      "fingerprintPaths": ["onyx/tools/evaluation", "scripts/evaluate_pid.sh"],
      "resources": ["simulator"],
      "timeoutSeconds": 600,
      "outputLimitBytes": 4000
    },
    "reliability.check": {
      "command": "bash",
      "args": ["onyx/tools/reliability/check.sh"],
      "timeoutSeconds": 300
    }
  },
  "workflow": [
    { "id": "edit", "agent": "Make one scoped code change, commit it, then resume." },
    { "id": "reset", "run": "reset.clean" },
    { "id": "evaluate", "run": "evaluation.run", "metric": true },
    { "id": "check", "run": "reliability.check", "guardrail": true }
  ]
}

experimentPolicy.mode: "single_candidate" tells workers to make one measured candidate per workflow. Short diagnostics are fine, but broad grid, sweep, probe, or scratch tuning scripts should be treated as policy violations unless setup explicitly changes the policy.

Metric Tool

The required metric step must print exactly one primary metric line:

METRIC tracking_error=0.18

Example tool:

#!/bin/bash
set -euo pipefail

./scripts/evaluate_pid.sh > /tmp/onyx-pid-results.txt
tracking_error=$(awk '/tracking_error/ {print $2}' /tmp/onyx-pid-results.txt)
echo "METRIC tracking_error=$tracking_error"

If the tool exits nonzero, times out, emits no primary metric, or emits the wrong metric name, the workflow attempt is recorded as failed. The required metric tool must also declare non-empty fingerprintPaths. Directories expand recursively to sorted tracked regular files. Onyx rejects missing, untracked, escaping, or symlinked paths, then fingerprints committed git contents together with the metric step and complete normalized tool configuration. Changing those inputs creates a separate evaluation revision instead of mixing incomparable results.

Guardrail Tools

Use guardrail: true steps for tests, typechecks, lint, safety checks, or anti-gaming checks that should run after a valid metric.

#!/bin/bash
set -euo pipefail

bun run typecheck
bun test

If a required guardrail fails, the experiment is recorded as checks_failed. Optional workflow steps can fail without determining the experiment status.

Manual Tool Runs

Use declared tools for smoke checks before Research:

onyx-worker tools run evaluation.run
onyx-worker tools run reliability.check

Manual tool runs never create workflow state, never satisfy workflow steps, and never create a local experiment attempt.

How to Change Setup

Ask the orchestrator to revise setup when you need to change agent behavior, measurement, reset, tools, resources, or hard constraints. Examples:

You want to change	Setup artifact
Goal, metric, scope, workflow tools, resources, checks	`onyx/setup.json`
Durable project research spec	`onyx/onyx.md`
Helper implementation	`onyx/tools/*`

After changing setup intent, tell the orchestrator:

/onyx Stop the current session, revise setup with these changes, validate it, and restart Research

Protected During Research

Hypothesis workers should not modify:

onyx/onyx.md;
onyx/setup.json;
onyx/validation.json;
onyx/tools/*.

The orchestrator can improve these files by revising setup and validating before starting more hypothesis workers.

Getting Started

Tutorials

Concepts

Platform

Understand setup workflows and tools

`onyx/setup.json`: Setup Policy

Metric Tool

Guardrail Tools

Manual Tool Runs

How to Change Setup

Protected During Research

​onyx/setup.json: Setup Policy

​Metric Tool

​Guardrail Tools

​Manual Tool Runs

​How to Change Setup

​Protected During Research

`onyx/setup.json`: Setup Policy

Metric Tool

Guardrail Tools

Manual Tool Runs

How to Change Setup

Protected During Research