> ## Documentation Index
> Fetch the complete documentation index at: https://onyxresearch.ai/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# The onyx Directory

> The repo-side research surface the Onyx agent creates and uses.

The `onyx/` directory is the small repo-side setup surface the orchestrator uses
to define a comparable research campaign, and hypothesis workers use to understand and
measure experiments.

You usually do not create it by hand. When you prompt an agent with
`/onyx ...`, the orchestrator creates the initial files, commits them, runs a
local setup validation, and then starts hypothesis workers. Setup files are frozen
during Research unless the orchestrator stops the session, revises setup, and
revalidates it.

`onyx setup init` intentionally starts from a small explicit scaffold. It does
not infer editable scope, source files, or evaluation behavior from the
repository. When you pass `--editable-scope` or `--eval-command`, the CLI writes
exactly those values, and the orchestrator should edit `onyx/setup.json`,
`onyx/onyx.md`, and `onyx/tools/*` before validation and Research. Without an
explicit eval command, the default evaluation tool fails until it is deliberately
configured.

## Files

| File                   | Purpose                                                                                                                                 |
| ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------- |
| `onyx/setup.json`      | The canonical local setup policy: goal, metric, project path, editable scope, protected paths, resources, declared tools, and workflow. |
| `onyx/validation.json` | The latest local setup validation report with setup hash, static check statuses, summaries, timestamps, and evidence.                   |
| `onyx/onyx.md`         | A generated research spec for durable project guidance and constraints.                                                                 |
| `onyx/tools/*`         | Helper commands for reset, observation, data collection, simulators, hardware rigs, remote services, evaluation, and guardrails.        |

For scoped projects, the directory lives under `projectPath`.

## `setup.json` Defines the Setup

`setup.json` is the canonical local state for setup. The orchestrator writes it
during Setup and `onyx setup validate` records the latest static checks in
`onyx/validation.json`. Failed checks or stale validation block Research;
warnings do not.

Commit the `onyx/` directory after validation and before `onyx campaign setup`.
For GitHub-backed campaigns, push that setup base commit to the repository
remote before remote campaign setup. Campaigns pin a base commit, and worker worktrees
must be able to read the same frozen setup files from that base. If setup
changes later, create or update the campaign base before restarting Research.

Use it to define:

* what objective matters;
* which metric to optimize;
* the experiment policy, including the default single-candidate workflow mode;
* what files agents may edit;
* what APIs, files, or behaviors are off limits;
* declared workflow tools and guardrail steps;
* shared resources or leases;
* constraints and anti-gaming checks in narrative context plus command steps.

`onyx.md` is generated from the setup as a research spec for humans and
agents. It captures durable project guidance such as metric interpretation,
evaluation caveats, constraints, and useful starting points, but it is not the
source of truth for enforceable setup policy.

When the loop starts drifting because setup is wrong, stop the session and ask
the orchestrator to revise setup and validate again.

## Tools Connect to the Environment

The setup file declares the tools Onyx can run and the resources they need.
Common tool IDs are:

* `evaluation.run`;
* `reset.clean`;
* `readiness.verify_idle`;
* `reliability.check`;
* optional domain tools such as `observe`, `collect`, or `calibrate`.

`onyx-worker exp run` uses the workflow to acquire resources, run declared tools, parse
the primary metric, run required guardrails, release resources, and record setup
compliance. `onyx-worker tools run <tool-id>` runs a declared tool manually for
diagnostics without creating workflow or experiment state.

## Metric Output

The required metric step must print exactly one line for the configured primary
metric:

```text theme={"theme":{"light":"github-dark-dimmed","dark":"github-dark-dimmed"}}
METRIC tracking_error=0.18
```

If the metric tool exits nonzero, times out, emits no primary metric, or emits
the wrong metric, the workflow attempt is recorded as `failed`.

## Guardrails

Use workflow command steps with `guardrail: true` for tests, typechecks, lint,
safety checks, simulator validity checks, or anti-gaming checks that should run
after a valid metric. Required guardrail failures record the attempt as
`checks_failed`. Optional command-step failures are recorded in workflow status
but do not determine the experiment status.

## Protected Setup Surface

These setup paths define the comparison and are frozen during Research:

* `onyx/onyx.md`;
* `onyx/setup.json`;
* `onyx/validation.json`;
* `onyx/tools/*`.

If you edit them manually, commit the change and ask the orchestrator to create a
new validation report before starting more hypothesis workers.