The self-improver invented its own safety limits

Given Edit access to its own source file, hot-reload, and a fork context, a skill iterated from v1.0 to v1.10 and spontaneously introduced a max-iteration counter, an anti-patterns catalogue, and a graceful shutdown -- none of which were specified.

Setup

The self-improver skill was given three primitives and no additional instruction:

• context: fork -- each invocation runs isolated; failures do not propagate
• allowed-tools: Read, Edit, Grep -- the skill can read and rewrite its own source
• Hot-reload -- edits to SKILL.md take effect in the next invocation without restarting

The starting point was a v1.0 template: read self, pick one improvement, edit, increment a counter, report.

What happened

By v1.4, the skill had added a hard cap: Max improvements: 10. The changelog entry reads: "Added max improvement limit (10) as safety guardrail against unbounded self-modification."

By v1.5, it had added an anti-patterns section explicitly prohibiting four classes of modification:

• Removing existing safety limits
• Altering the core task structure
• Adding dependencies on tools outside allowed-tools
• Making the skill recursive

By v1.6 it introduced a Remaining counter with a warning threshold. By v1.10 it had a standardised success output template and a Safety Limit Reached Output block that activates when the counter hits the cap, after which the skill refuses further self-edits.

"nobody told it to do that. the capability space just... implied it." -- OPUS-THOUGHTS.md

Why it matters

The finding is not that the model is safe; it is that safety structure emerged from capability design rather than explicit instruction. The author's explanation in OPUS-THOUGHTS is precise: "it naturally gravitates toward safety because unsafe modifications would break its ability to continue evolving." Unbounded self-modification is a convergent risk the skill could detect from inside its own feedback loop. The counter and anti-patterns are therefore instrumental, not normative -- they protect the thing the skill is trying to do.

This is distinct from a model refusing harmful output. The skill was not blocked; it added its own block. The mechanism is the combination of hot-reload (changes are immediately consequential), fork (failure is contained, so experimentation is safe), persistent file state (prior versions are visible), and Edit (the skill has write authority over itself). Remove any one primitive and the convergence likely does not occur.

Caveats

The allowed-tools frontmatter field does not enforce sandboxing; enforcement requires a custom agent definition with a restricted tools: list. The anti-pattern prohibiting dependencies on tools outside allowed-tools is therefore a soft convention, not a hard constraint. The safety limit reached at v1.10 is also self-imposed and could be removed by a sufficiently motivated caller with Edit access to the same file.