> ## Documentation Index
> Fetch the complete documentation index at: https://www.onyxresearch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Understand setup workflows and tools

> Learn how the Onyx agent creates, uses, and validates repo-side setup workflows.

When you start with `/onyx ...`, the orchestrator creates the repo-side setup
files it needs to run comparable parallel research. You can review the setup
before Research starts; changing setup during Research should stop the session,
revise setup, and run validation again.

## `onyx/setup.json`: Setup Policy

`setup.json` is the canonical local setup policy for the orchestrator and
hypothesis workers. It defines what agents optimize, how measurement runs, what
files are in scope, which setup paths are protected, what resources are leased,
which tools are available, and the linear workflow for one experiment attempt.

The orchestrator creates the first version and `onyx setup validate` writes the
latest check results to `onyx/validation.json`. During Research, hypothesis
workers treat setup files as frozen. If setup is wrong, stop the session and
revise it.

Example:

```json theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
{
  "schemaVersion": 2,
  "goal": "Minimize tracking error for the arm controller without increasing overshoot.",
  "experimentPolicy": {
    "mode": "single_candidate",
    "maxDiagnosticSeconds": 30
  },
  "projectPath": "",
  "scope": {
    "editable": ["src/control"],
    "protected": [
      "onyx/setup.json",
      "onyx/validation.json",
      "onyx/onyx.md",
      "onyx/tools/"
    ]
  },
  "metric": {
    "name": "tracking_error",
    "unit": "error",
    "direction": "minimize"
  },
  "resources": {
    "simulator": { "slots": 1, "description": "Controller simulator" }
  },
  "tools": {
    "reset.clean": {
      "command": "bash",
      "args": ["onyx/tools/reset.sh"],
      "resources": ["simulator"],
      "timeoutSeconds": 60
    },
    "evaluation.run": {
      "command": "bash",
      "args": ["onyx/tools/evaluation/run.sh"],
      "fingerprintPaths": ["onyx/tools/evaluation", "scripts/evaluate_pid.sh"],
      "resources": ["simulator"],
      "timeoutSeconds": 600,
      "outputLimitBytes": 4000
    },
    "reliability.check": {
      "command": "bash",
      "args": ["onyx/tools/reliability/check.sh"],
      "timeoutSeconds": 300
    }
  },
  "workflow": [
    { "id": "edit", "agent": "Make one scoped code change, commit it, then resume." },
    { "id": "reset", "run": "reset.clean" },
    { "id": "evaluate", "run": "evaluation.run", "metric": true },
    { "id": "check", "run": "reliability.check", "guardrail": true }
  ]
}
```

`experimentPolicy.mode: "single_candidate"` tells workers to make one measured
candidate per workflow. Short diagnostics are fine, but broad grid, sweep,
probe, or scratch tuning scripts should be treated as policy violations unless
setup explicitly changes the policy.

## Metric Tool

The required metric step must print exactly one primary metric line:

```text theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
METRIC tracking_error=0.18
```

Example tool:

```bash theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
#!/bin/bash
set -euo pipefail

./scripts/evaluate_pid.sh > /tmp/onyx-pid-results.txt
tracking_error=$(awk '/tracking_error/ {print $2}' /tmp/onyx-pid-results.txt)
echo "METRIC tracking_error=$tracking_error"
```

If the tool exits nonzero, times out, emits no primary metric, or emits the
wrong metric name, the workflow attempt is recorded as `failed`.

The required metric tool must also declare non-empty `fingerprintPaths`.
Directories expand recursively to sorted tracked regular files. Onyx rejects
missing, untracked, escaping, or symlinked paths, then fingerprints committed
git contents together with the metric step and complete normalized tool
configuration. Changing those inputs creates a separate evaluation revision
instead of mixing incomparable results.

## Guardrail Tools

Use `guardrail: true` steps for tests, typechecks, lint, safety checks, or
anti-gaming checks that should run after a valid metric.

```bash theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
#!/bin/bash
set -euo pipefail

bun run typecheck
bun test
```

If a required guardrail fails, the experiment is recorded as `checks_failed`.
Optional workflow steps can fail without determining the experiment status.

## Manual Tool Runs

Use declared tools for smoke checks before Research:

```bash theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
onyx-worker tools run evaluation.run
onyx-worker tools run reliability.check
```

Manual tool runs never create workflow state, never satisfy workflow steps, and
never create a local experiment attempt.

## How to Change Setup

Ask the orchestrator to revise setup when you need to change agent behavior,
measurement, reset, tools, resources, or hard constraints.

Examples:

| You want to change                                     | Setup artifact    |
| ------------------------------------------------------ | ----------------- |
| Goal, metric, scope, workflow tools, resources, checks | `onyx/setup.json` |
| Durable project research spec                          | `onyx/onyx.md`    |
| Helper implementation                                  | `onyx/tools/*`    |

After changing setup intent, tell the orchestrator:

```text theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
/onyx Stop the current session, revise setup with these changes, validate it, and restart Research
```

## Protected During Research

Hypothesis workers should not modify:

* `onyx/onyx.md`;
* `onyx/setup.json`;
* `onyx/validation.json`;
* `onyx/tools/*`.

The orchestrator can improve these files by revising setup and validating before
starting more hypothesis workers.