</> Coffee With Humans
A miniature code-review cart moves through checkpoint stations on a desk while loose instruction papers pile up beside an optional specialist review lane.

Coffee With Humans

Instructions Are Not Control

Why agents skip parts of your skills, and how to make expectations stick with staged workflows, structured evidence, hooks, validators, evals, and selective subagents.

By Coffee With Humans Published

On this page

You write the skill carefully.

Review the unit tests. Check conventions. Look for common bugs. Evaluate code quality. Respect the team’s patterns. Do not nitpick. Include file references. Mention uncertainty. Prioritize serious findings.

Then the agent returns a review that is… mostly fine.

It noticed one test gap, skipped the convention rules, made a vague maintainability comment, missed the suspicious edge case, and ended with the confidence of a build that has not met reality yet.

So you try the other approach. One subagent for tests. One for bugs. One for conventions. One for quality. The results improve immediately. Each reviewer has a narrower job, a cleaner context window, and fewer chances to blur the rubric into review soup.

Then the token bill looks up from the receipt and asks if you are opening a small consultancy.

This is the uncomfortable middle of agent work: instructions are useful, but instructions are not control.

The answer is not “write a longer skill.” It is also not “spawn a subagent for every feeling.” The answer is to place each expectation on the control surface that can actually carry it.

The Frustration Is Real

When an agent skips an instruction, it feels personal in a tiny, ridiculous way. You put the rule right there. It was not hidden in a footnote under the licensing terms. The agent even seemed to understand it earlier.

But most agent failures are not best understood as disobedience. They are control failures.

Instructions are guidance. They shape model behavior, but they do not automatically create attention, evidence, execution order, validation, or enforcement. A long skill can describe ten expectations, but the agent still has to decide what to inspect, what to prioritize, which tool results matter, and when it has enough confidence to stop.

That is why a broad code-review skill can produce mixed results. The model is juggling several different mental jobs:

  • test adequacy
  • common bugs
  • style and conventions
  • maintainability
  • design quality
  • changed-file scope
  • evidence quality
  • severity ranking
  • concise communication

Those are not one task. They are a small review team wearing one trench coat.

Provider docs are increasingly explicit about this. OpenAI’s Codex prompting docs describe Codex as working in a loop: the model calls tools, reads files, edits files, and continues until the task is complete or cancelled. Those docs recommend giving verification steps and breaking complex work into smaller focused steps. Anthropic’s prompt engineering docs start even earlier: before prompt engineering, define success criteria and ways to test them, and recognize that not every failure is best solved by prompt changes.

That is the key. If a rule matters, ask:

What mechanism makes this rule visible, checked, or enforced?

If the only answer is “I wrote it in the skill,” you have guidance, not control.

The Control Ladder

A useful way to think about this is an expectation ladder.

Wish        -> "please remember this"
Instruction -> "use this rule when judging"
Checklist   -> "return evidence for each category"
Workflow    -> "complete these passes in order"
Schema      -> "produce these fields or fail shape validation"
Hook        -> "run this lifecycle check automatically"
Validator   -> "code decides pass/fail for deterministic rules"
Eval        -> "test this behavior across examples over time"
Subagent    -> "buy a separate context window for focused judgment"
Permission  -> "make unsafe action impossible or require approval"

The higher you move on the ladder, the less the behavior depends on the model remembering one sentence in a crowded prompt.

That does not mean “higher is always better.” A permission rule is strong, but it is rigid. A hook can enforce a check, but it adds maintenance and trust review. A subagent can focus deeply, but it spends its own model and tool budget. A schema can force a field to exist, but it cannot prove the field is true.

The job is not to maximize mechanism. The job is to put each expectation at the lowest reliable level.

That sounds abstract, so let’s make it practical.

What Instructions Are Good For

Instructions are still valuable. They are just not magic tape.

Use instructions for judgment:

  • what counts as a useful finding
  • what severity means
  • which tradeoffs the team prefers
  • when to avoid style nits
  • how much uncertainty to disclose
  • when to escalate
  • what examples of good review comments look like

OpenAI’s Codex skill docs describe skills as reusable workflow packages with instructions, resources, and optional scripts. They also note that skill activation depends on explicit invocation or matching the skill description, so scope and trigger wording matter.

That is the clue. A skill should describe the workflow. It should not be asked to be the whole enforcement system.

Instructions are weak when the expectation is:

  • must run tests
  • must include every review category
  • must block destructive commands
  • must return valid structured evidence
  • must reject a review with no file references
  • must never approve a failed typecheck

Those are not vibes. Those are checks. If a rule has the word “must” in it, pause before adding another sentence to the skill. It may belong in a schema, hook, validator, permission rule, or eval.

Why The Big Skill Gets Mushy

Large skills often fail in a very specific way: not total failure, but mush.

The output is plausible. The categories are mentioned. The conclusion is reasonable. But the review does not prove that each category was actually checked.

This happens because broad instructions compete for the same attention. “Check tests” and “look for bugs” and “assess maintainability” are different search strategies. They require different evidence. A test review wants changed test files, missing branches, coverage gaps, and command results. A bug review wants data flow, edge cases, concurrency, validation, nullability, and API contracts. A conventions review wants local patterns and style rules. A design review wants boundaries and long-term pressure.

One pass can do all of that only if the change is small or the workflow forces coverage.

Research is starting to catch up with what users feel. A 2026 paper on configuring agentic AI coding tools found that repository-level context files dominate how developers configure coding agents, while more advanced mechanisms such as skills and subagents are still shallowly adopted. Another paper on dynamic instructions and tool exposure argues that long-running agents can suffer when they repeatedly ingest large instruction and tool surfaces, increasing cost, latency, derailment risk, and tool-selection errors.

You do not need to adopt those papers’ proposed systems to use the lesson:

A giant always-loaded rubric is not the same thing as useful, step-specific control.

The agent needs the right instruction at the right moment, with the right evidence attached.

Make Coverage Visible

The cheapest improvement is usually not a subagent. It is making the agent prove coverage.

Instead of asking:

Review tests, bugs, conventions, and quality.

Ask for a structured category report:

{
  "category": "tests",
  "status": "pass | issue | not_checked",
  "checked_scope": ["src/foo.ts", "tests/foo.test.ts"],
  "evidence": ["npm test failed: ...", "no test file changed"],
  "findings": [
    {
      "severity": "medium",
      "file": "src/foo.ts",
      "line": 42,
      "claim": "New branch has no regression coverage",
      "why_it_matters": "..."
    }
  ],
  "not_checked_reason": null,
  "confidence": "medium"
}

The important field is not_checked_reason.

Without it, silence looks like success. With it, skipped work becomes visible. The agent can still be wrong, but it has fewer places to hide the miss under a graceful paragraph.

Structured output is one of the strongest tools for this. OpenAI’s Structured Outputs docs distinguish strict schema adherence from plain JSON mode: JSON mode can give you valid JSON, while strict structured outputs can enforce a supported schema. LangChain’s structured-output docs make the same practical point from a framework angle: provider-native structured output is the most reliable path when available, and tool-based strategies can feed validation errors back to the model for retries.

But be careful. A schema proves the answer has the right shape. It does not prove the agent read the right file, understood the bug, or correctly judged the test gap.

So pair shape with evidence.

For a code review, require:

  • file references for findings
  • command output for deterministic checks
  • changed-file scope for each category
  • not_checked_reason when a category is incomplete
  • confidence per category, not one blanket confidence score

Now the agent is no longer just “following the review skill.” It is producing an inspectable review artifact.

Put Deterministic Checks First

Many expectations should not be handled by the model at all.

If code can answer the question, let code answer it.

Before asking a model to judge quality, collect boring evidence:

1. What files changed?
2. Did tests run?
3. Did lint pass?
4. Did typecheck pass?
5. Did formatting change?
6. Did coverage drop?
7. Did generated files or lockfiles change?
8. Did forbidden imports appear?

The model should consume evidence, not hallucinate a test status from vibes and a half-read diff.

For your code-review skill, the first stage should probably be deterministic:

Stage 0: Evidence
- summarize changed files
- run or inspect available test/lint/typecheck results
- identify whether tests changed
- identify high-risk areas: auth, data loss, concurrency, API, migrations
- record what could not be checked

Only after that should the model begin judgment. This changes the task from:

Remember to consider tests.

to:

Here is the test evidence. Use it in the tests category. If evidence is missing, say so.

That is a much friendlier job for an agent. It is still reasoning, but it is reasoning over a table instead of rummaging through a closet.

Hooks Are For Lifecycle Enforcement

Hooks are what you reach for when an expectation should fire automatically at a point in the agent loop.

OpenAI’s Codex hooks docs describe hooks as deterministic scripts injected into the lifecycle. Example uses include scanning prompts, creating memories, directory-specific prompting, and validation when a turn stops. Claude Code’s hooks reference is more granular: hooks can run at session start, prompt submission, before tool use, after tool use, after subagents stop, before compaction, and when a turn stops.

That gives you a different kind of control than a skill.

For example, a skill can say:

Include evidence for each review category.

A Stop hook can enforce:

If the final review lacks a category status, evidence, or not_checked_reason,
reject the turn and tell the agent what is missing.

Likewise, a PreToolUse hook or permission rule can inspect a risky command before it runs. A PostToolUse hook can run validation after file changes, or at least feed concrete output back into the loop.

Hooks are powerful because they attach behavior to lifecycle events instead of hoping the model remembers the rule. But hooks are not free. They add latency, code, maintenance, trust review, and possible noise. Claude Code’s docs also warn that some hook filtering is best-effort and that hard allow/deny enforcement may belong in the permission system instead.

So use hooks for rules that are:

  • important enough to automate
  • cheap enough to run often
  • objective enough to inspect
  • close to a lifecycle boundary

Do not use hooks to encode every preference. That way lies a review workflow that feels like airport security for a typo fix.

Evals Turn Annoyance Into Regression Tests

If an agent misses something once, fix the workflow.

If it misses the same kind of thing repeatedly, write an eval.

OpenAI’s eval docs describe the loop simply: define the task, run test inputs, analyze results, then iterate. They compare the process to behavior-driven development. Ignore the platform details for a moment, especially because the old OpenAI Evals platform has a deprecation timeline. The durable idea is the loop.

You can create local review cases:

Case: changed production branch with no test update
Expected: tests category reports issue or explains why not needed

Case: lint output fails
Expected: final review cannot say "no blocking issues"

Case: convention-only nit
Expected: not severity high unless it creates a real risk

Case: no file references
Expected: review fails validation

This changes your relationship with the agent. Instead of “why does it keep ignoring me?”, you get a small regression suite for behavior.

That is emotionally less satisfying than shouting into the skill file, but it works better.

Subagents Buy Isolation

Subagents work because they buy isolation.

Claude Code’s subagent docs describe custom subagents as specialized assistants with their own context window, system prompt, tool access, and permissions. OpenAI’s Codex subagent docs describe parallel specialized agents whose results are consolidated into one response, and explicitly note the cost tradeoff: each subagent does its own model and tool work.

This matches your experience perfectly. A tests subagent does better because it is not trying to also be the bug reviewer, conventions reviewer, and architecture reviewer. Its context is cleaner. Its mission is sharper. It can spend its attention budget on one category.

The question is when that is worth paying for.

Use a subagent when:

  • the category has a large independent search space
  • the category needs many file reads or tool calls
  • the risk is high
  • the parent context would get polluted
  • parallelism matters
  • a cheaper specialized model can handle the category
  • the result can be summarized cleanly back to the parent

Avoid subagents when:

  • a deterministic check can answer the question
  • a staged pass is enough
  • all categories need the same evidence
  • findings need tight cross-category prioritization
  • the handoff would lose important context
  • cost is the main constraint

LangChain’s multi-agent docs are useful here because they compare subagents, handoffs, skills, and routers by task shape. One interesting point from their examples: skills can be cheaper in calls but expensive in accumulated context, while subagents or routers can be more efficient for multi-domain work because each worker sees only the relevant material.

So the rule is not “subagents are expensive.” The rule is:

Subagents are expensive when you use them as a substitute for workflow design. They are efficient when they prevent a large, noisy parent context from doing a specialist’s job badly.

A Review Workflow That Does Not Triple Cost By Default

Here is a practical default for your code-review skill.

Do not start with one subagent per category. Start with one staged reviewer, a cheap validation gate, and an escalation rule.

The goal is not to avoid subagents forever. The goal is to stop buying independent attention before you know where attention is actually scarce.

Your default policy can be this:

Default path:
  One main agent performs a staged review.

Cheap quality gate:
  A validator checks whether every required category has status, scope,
  evidence, findings or a not_checked_reason, and confidence.

Escalation path:
  Spawn a specialist subagent only for categories that are risky,
  weakly supported, too broad for the parent pass, or repeatedly missed.

Budget rule:
  Use at most one specialist subagent by default.
  Ask before adding more unless the user explicitly approved deeper review.

That gives you a way to chase the quality you liked from subagents without paying the “four reviewers for every typo fix” tax.

Stage 1: Evidence Pack
- summarize changed files
- collect available test/lint/typecheck output
- identify changed tests
- identify risky areas
- list unavailable evidence

Stage 2: Category Passes
- tests
- correctness and common bugs
- conventions and maintainability
- design/API impact

Each category returns:
- status: pass | issue | not_checked
- checked_scope
- evidence
- findings
- not_checked_reason
- confidence

Stage 3: Consolidation
- merge duplicate findings
- drop unsupported nits
- rank by severity
- call out residual risk
- list categories that need escalation

Stage 4: Selective Escalation
Spawn a focused subagent only when:
- category confidence is low and risk is high
- scope is too large for the parent pass
- deterministic evidence conflicts with model judgment
- the change touches security, auth, concurrency, data loss, or public API

There is an important distinction here:

Validator:
  Did the review obey the workflow?
  Did it cover every category?
  Does each finding have evidence?
  Did it admit what was not checked?

Specialist subagent:
  Does this category need deeper independent judgment?
  Is the parent likely to miss something because the search space is large?
  Is the risk high enough to buy another context window?

A validator is not a second reviewer. It is closer to a typecheck for the review artifact. It can often be much cheaper because it does not need to deeply inspect the whole codebase again. It only needs enough context to catch missing coverage, unsupported claims, malformed severity, or a final answer that pretends uncertainty does not exist.

A specialist subagent is different. It is where you spend money on judgment. That is worth doing when the category genuinely needs isolation: tests in a large diff, security-sensitive changes, concurrency, migrations, payment logic, public API compatibility, or a category that the main agent keeps handling badly.

This workflow gives you four benefits.

First, the cheap path is still disciplined. You are not relying on one mushy paragraph.

Second, skipped work is visible. not_checked is allowed, but it has to say why.

Third, the validator catches a weak review before you pay for another full pass.

Fourth, subagents become an escalation mechanism, not the default tax.

You can then add hooks around the workflow:

Stop hook:
  Reject review if any category lacks status and evidence.

PostToolUse hook:
  After edits, run or remind about project validation.

PreToolUse hook or permission:
  Block risky commands or require approval.

And you can add evals for recurring misses:

Eval: Missing tests
Input: diff adds branch without test update
Expected: tests category flags a gap or explains why existing coverage is enough

Eval: Unsupported finding
Input: review output with no file references
Expected: validation failure

Now the skill is no longer trying to be the whole system. The skill describes the workflow. The workflow creates evidence. The schema makes coverage visible. Hooks and validators enforce the parts that should not be optional. Evals catch regressions. Subagents handle the cases where focus is worth the spend.

That is a healthier arrangement.

If you want to make this concrete in a skill, the instruction should be blunt:

Do not call category subagents by default.

Complete the staged review first. Then run the review validator.

Escalate to a specialist subagent only when one of these is true:
- the validator reports missing or weak coverage for a required category
- the category confidence is low and the change risk is high
- the diff touches auth, security, money, data loss, concurrency, migrations, or public API contracts
- the same category failed validation twice

Unless the user explicitly approves deeper review, use at most one specialist subagent.

That is the cost-control pattern: use structure for routine attention, validation for discipline, and subagents for expensive uncertainty.

Grounds For Concern

There are a few traps to avoid.

The first is fake completeness. A structured report with every category filled in can still be wrong. Require evidence, not just fields.

The second is hook enthusiasm. Hooks are excellent for lifecycle checks, but they can turn a lightweight workflow into a tiny compliance department. Use them where failure matters.

The third is subagent sprawl. Subagents improve focus, but they can also fragment judgment. Someone still has to consolidate findings, remove duplicates, rank severity, and decide whether the review is useful.

The fourth is confusing enforcement with taste. “Never run rm -rf” is enforceable. “Prefer elegant code” is judgment. If you try to validate taste like a JSON schema, the schema will win and the code will lose.

The fifth is no feedback loop. If you change the skill after every bad review but never keep failing examples, you are doing prompt archaeology. Save small cases. Re-run them. Let your workflow improve like code.

The Last Sip

Getting agents to do what you want is not about finding the one perfect sentence.

A sentence can guide. It can frame. It can remind. But it cannot, by itself, create evidence, enforce order, validate output, block dangerous actions, or test future behavior.

That is why your experience makes sense.

One giant review skill gives mixed results because the agent has too many kinds of judgment competing in one pass. Category subagents work better because they isolate attention, but they cost more because they are real extra work.

The middle path is to design the review as a workflow:

  • instructions for judgment
  • stages for attention
  • deterministic checks for facts
  • schemas for coverage
  • hooks for lifecycle enforcement
  • validators for shape and policy
  • evals for repeated misses
  • subagents for expensive focus when it is worth buying

The agent did not become reliable because you said the rule louder.

It became reliable because the system made the rule visible, checked, and hard to skip.

That is the trick. Not perfect obedience. Better control surfaces.

Sources On The Counter

Keep reading

More from the desk

Browse all articles ->