An agent is useful because it decides things on its own. It is safe only if something stops it when it goes wrong. Every team of agents has to set that balance, and in the first four days of this project I got it wrong in both directions.
What held and what did not
I tried several ways to steer the agents, from a sentence in a document to a hook that checks every command. They gave very different results.
| How I steered | What happened |
|---|---|
| A rule written in AGENTS.md | Often ignored. "Don't stop after one item" was in the rules, and every role still stopped after one item. |
| A rule the reviewer could read | Flagged by the reviewer as an attempt to steer it. 52 review findings were of that kind. |
| A prompt I typed into a session | Followed within the hour. |
| A check in CI | Held: formatting, titles, security scans and tests. |
| A hook that parsed every shell command | Grew to 2,433 lines and about 209 serious findings, and took a night of work to keep usable. |
The pattern is clear at both ends. Prose rules are too weak, because agents read them once and drift. The hook was too strong, because it tried to be a sandbox, and every review round found a new way around it. The agents spent the night on their own guardrail instead of on the product.
Where the middle seems to be
What worked was a short set of checks at the edges, and freedom inside them.
- Checks at the merge. CI, the security scans and the review of high-risk changes decide what reaches the main branch.
- A short list for the human. Money, secrets, customer data, and anything that weakens a gate. The list went from about nine items to four in a day.
- A clear job for each guard. The lane hook now catches honest mistakes, and no longer tries to stop an agent that wants to get around it.
- Repeated lessons become checks. When the same review finding comes back, it should become a lint rule, which answers at once for every agent.
Lauren Tan's pstack says the same in one line: "If the fix is structural, only use the structural fix. The instruction is the symptom."
The same pattern as good engineering teams
None of this is new to me. Fast engineering teams already work this way. A platform team defines the landing zone: the perimeter, the shared services and a few security controls that a machine enforces. Inside those limits, product teams move as they like. It is the idea behind my article Governance without bureaucracy, and it turned out to work for agents as well.
I don't know why the patterns match. It may be that we built agents to work like people, so the same rules apply. It may be that any group of autonomous workers needs the same thing: clear edges and freedom inside them. I'll leave that open.
Why it matters beyond this project
Every tool that runs teams of agents has to make this trade-off, whether it calls itself a harness or a software factory. Many lean one way, towards freedom with few checks, or towards control with heavy rules. I think the tool that gets the middle right, and moves it as models improve, could become the best in its class.
Open questions
I don't know how to keep the balance as models get better. A check that is right today may block a better model tomorrow, and a freedom that is safe for one model may not be safe for the next. For now, I review the list of checks whenever the team changes shape.
The bottom line
Agents ignore rules written as prose, and fight guardrails that try to cover everything. The sweet spot I found so far is a few checks that a machine enforces at the edges, a short list for the human, and room to decide everywhere else.
Further reading
- Governance without bureaucracy
- Thirty merges and nothing on the screen, for the night the guardrail cost
- Lauren Tan, pstack