Scout
DocsBlogToolsContact
Building on Scout03 / 03~3 min

12 · Adding a harness

The full integration checklist, worked on the Devin ACP PR.

View MD

Goal

The full checklist for making a coding agent a first-class Scout harness — worked on the Devin ACP integration, which landed exactly this way (PR #940↗).

You need

11 done; a checkout of the repo. Read agent-integration-contract.md first — it is the contract this page operationalizes.

The idea

The adapter is the easy part. ACP-capable agents share one transport adapter; the real work is the closed-union surface — roughly fifteen hand-maintained lists across protocol, runtime, CLI, web, and desktop that must all learn the new name. Missing one produces silent drift, not a compile error.

Walk it

Step 1 — probe the real binary first

Do this before writing code; it settles the design:

devin acp --help # flags: is --model a launch arg? in-band auth?
devin models # real model ids and context windows
# then drive the ACP handshake and read initialize/session capabilities

What the probe decides: requireAuth (Devin advertises only browser auth — undrivable headlessly — so false, relying on stored credentials), launch-time versus in-band model selection, and session lifecycle (loadSession? resume? close?).

Step 2 — sweep all three spellings

A harness exists under three identifier forms. For Devin: devin (harness id), devin_acp (transport literal), devin-acp (adapter type). Every list, regex, and registry must be swept for all three — sweeping only the transport literal is how the web pairing registry got missed.

Step 3 — the union surface

Touch each of these (the current list, verified during the Devin work):

  • packages/protocol: AGENT_HARNESSES, SCOUT_LAUNCHABLE_HARNESSES, AgentEndpointTransport, DeliveryTransport, RelayRuntimeTransport, the runtime catalog JSON + generated file.
  • packages/agent-sessions: the adapter directory (adapter, spec, test, index), local/index.ts registry, auth/domains.ts.
  • packages/runtime: ManagedAgentHarness, DirectLocalAgentTransport, CardlessSessionSpawnTransport, acp-agent-invocation maps, the invocation wrapper, local-agents.ts dispatch sites (harness mapping, shutdown, online/binary checks, both invoke paths), setup.ts, onboarding.ts, cardless spawn, record strings/reader, sync service, pairing attribution, coding-agent-host.ts detection, conversation projection aliases.
  • CLI/web/desktop: --default-harness validation lists (both CLIs), scout-mcp label regex, session-start adapter map, both pairing registries (desktop and web server — they are separate copies), slack manifest, machine-update markers, HarnessMark labels.

tsc catches Record<> exhaustiveness; it does not catch Set/array literals or regexes — those were already stale for earlier harnesses.

Step 4 — verify like a reviewer

npm --prefix packages/protocol run check
npm --prefix packages/runtime run check
bun test packages/agent-sessions/src/adapters/<name>-acp/
# plus the harness-catalog, runtime-catalog, host-detection, pairing tests

Then launch the real binary through the adapter once — a fake executable proves the plumbing; only a live session proves the integration.

How to tell it worked

  • scout runtimes (on a rebuilt suite) lists the harness with models.
  • scout ask --harness <name> routes and a real turn completes.
  • Host detection resolves the right sender inside its spawned shells.

Push further

  • Add parity tests so the next harness is a smaller diff: both pairing registries expose the same adapters, every launchable harness has a valid transport, every transport has dispatch/shutdown/serialization coverage.
  • Derive validation lists from SCOUT_LAUNCHABLE_HARNESSES instead of duplicating literals where the shape allows it.
  • Prefer official vendor host markers (DEVIN_AGENT-style) over internal codenames; guard the undocumented ones.

Where you end up

You now have the whole loop: operate the crew, route the work, read the records, and extend the runtime roster. The reference docs in README.md↗ go deeper on each noun.