The Grown-Up Rules for Plugging Cognition Into Everything

preamble

We used to wire electricity into everything. Now we’re wiring cognition into everything.

Agents, copilots, automated workflows – whatever you call them – are already touching code, money, customers, infrastructure, and law. We’re letting semi-autonomous systems act on our behalf in places where mistakes are very expensive.

Most of the industry is still acting like this is a toy.

This doc is the minimum bar for adulthood.

If you want to plug cognition into your company, you don’t get vibes and hype. You get constraints, logs, roles, and responsibility.

These are the minimum rules to be a grown-up about agents. If you’re below this bar, admit you’re still playing with matches.

rule 1 — no unlogged cognition

No ghost workers in core systems.

If an agent can:

• Read sensitive data
• Change state (code, infra, records)
• Touch money
• Talk to customers
• Affect compliance / reporting

Then every action must be logged:

• What it saw
• Tools / APIs it called
• What it changed
• Who approved, and when
• Timestamps, IDs, correlation IDs

No log = no trust.

If we can’t replay what your “smart system” did, it doesn’t belong anywhere near critical workflows.

rule 2 — humans own outcomes

Responsibility is not automatable.

Every agentic system has a named human steward.

That person:

• Signs off on what the system is allowed to do
• Understands the failure modes
• Is on the hook in incident reviews
• Can say “no” and dial autonomy back

There is no “the AI did it.” The stack ends on a human.

rule 3 — capability follows competence

No L1 brain gets L5 powers.

You don’t hand a med student solo brain surgery. You don’t hand a brand-new agent root access.

Define clear capability levels, e.g.:

• L1 – Observer: read-only, explains + summarizes
• L2 – Copilot: drafts outputs, no direct side-effects
• L3 – Operator: can act in low-risk, easily reversible domains
• L4 – Designer: can propose workflows + changes, gated by human approval
• L5 – Steward: designs systems, sets policies, supervises others

To move a system up a level, it has to pass evals on:

• Correctness
• Robustness under stress
• Interpretability (can we explain its behavior?)
• Blast radius + rollback

No more “well, the demo looked good so we gave it prod access.”

rule 4 — sandbox before surface area

Everything dangerous grows up in a box first.

Deployment should look like:

1. Sandbox phase

   • Synthetic or restricted data
   • Heavy logging
   • Adversarial tests

2. Shadow phase

   • Runs alongside humans on real data
   • Proposes actions, doesn’t execute
   • Human decisions + overrides are logged as training signal

3. Gradual rollout

   • Start with narrow scope / low blast radius
   • Feature flags, canaries
   • Instant rollback paths

If you “don’t have time” for this, you don’t have time for agents.

rule 5 — tight feedback or no autonomy

If we can’t tell when it’s going off the rails, it doesn’t run unattended.

Autonomy is not a yes/no; it’s bounded by how fast you can detect and recover from bad behavior.

For any unattended agent:

• Define SLOs / guard metrics (error rates, anomaly thresholds, financial deltas, etc)
• Define kill switches + rollback (who can hit them and how)
• Wire alerts to actual humans with enough context to act

No monitoring = no autonomy. “Set and forget” cognition is how you get surprise, expensive art projects in prod.

rule 6 — interpretability over vibes

We wrap black boxes in white-box workflows.

You will not get perfect “explainability” from LLMs. That’s fine.

For high-stakes use (money, safety, infra, compliance, brand), you still owe:

• Replayable traces (see rule 1)
• Visible decision paths:
  • Which tools it called
  • Which checks it passed/failed
  • What constraints it was running under
• Rationales tied to observable facts (“because X was true, we chose Y”)

We don’t need to peek inside the model’s weights. We do need to understand the container we put it in.

rule 7 — socio-technical, not tech-only

Culture is part of the runtime.

Agents don’t live “in isolation.” They live inside orgs with incentives, politics, and human failure modes.

Grown-up deployments answer:

• Who is trained to design, operate, and debug this?
• What rituals support it? (reviews, standups, postmortems)
• What incentives shape behavior? (“ship or die” vs “safe + fast”)
• What happens when someone says “this feels wrong”?

There is no tooling that can save an org whose culture punishes people for pulling the brake. You can’t tech your way out of cowardly leadership.

rule 8 — minimal necessary agency (principle of least agency)

Don’t summon a demigod when you just needed a calculator.

Just because you can jam a general agent into something doesn’t mean you should.

For every use case, ask:

• Can this be a narrow tool instead of a general “do anything” agent?
• Can we restrict:
  • Domains
  • Actions
  • Time budget
  • Data access
  • Downstream systems it can touch

We default to minimal necessary agency. Generality is an escalation, not a starting point.

rule 9 — fail loudly, recover gracefully

No silent corruption.

Bad things will happen. The question is whether you:

• Notice
• Contain
• Learn

Grown-up behavior:

• Detect anomalies quickly (rule 5)
• Page the right humans
• Preserve evidence: traces, diffs, environment
• Run blameless postmortems that include the agent’s behavior, the human decisions, and the guardrails that failed

Then you update:

• Evals
• Guardrails
• Docs/runbooks
• Training material
• Maybe the covenant itself

Every agent-related failure becomes training data for both humans and systems.

rule 10 — multi-horizon ethics

Optimize today without poisoning tomorrow.

You’re not just responsible for the next quarter.

Grown-up deployments consider:

• Now: speed, savings, quality
• Mid-term:
  • Skill atrophy (“are we forgetting how to do this without agents?”)
  • Organizational drift (“does this incentive people to cut corners?”)
  • Dependency risk (“what happens when this goes down?”)
• Long term:
  • Are we building systems that:
    • Can be audited?
    • Can be handed to the next generation without them needing a séance?
    • Concentrate power into opaque stacks?

You don’t have to be a saint. You do have to stop trading systemic integrity for a little line-go-up dopamine.

If you can’t meet these rules, say so out loud.

Just don’t pretend you’re doing “responsible AI.”