Counterfactual genealogy
A theory of recursive development should be able to lose. The central test compares lineages that inherit a modification with matched lineages in which it is reverted.
The basic fork
The central experiment is deliberately simple. Allow a system to generate and adopt modification X. Immediately after adoption, fork the lineage.
lineage R
lineage V
Everything possible should be matched across the two branches: factual information, compute, wall-clock budget, search budget, tool access, storage, evaluator calls, test distribution, and human contribution. The experimental question is whether inheriting X changes a preregistered property of the later improvement transition.
What to measure
- Probability of producing a valid later improvement.
- Quality and calibration of diagnosis.
- Search efficiency and validation cost.
- Generalization to held-out tasks and environments.
- Regression rate, negative transfer, and rollback performance.
- Evaluator independence and resistance to benchmark exploitation.
A particularly strong assay uses a neutral recipient. Let each branch generate a candidate later modification, then transplant those candidates into an otherwise matched neutral system. If changes produced by the retained lineage transfer better, that helps separate “better improver” from “better starting system.”
Controls that matter
- Full reversion: remove X while preserving all other matched state.
- Raw-facts control: preserve factual discoveries without preserving acquired procedure.
- Standardized-facts control: give both branches the same distilled information.
- Procedure-only control: retain acquired organizational method while equalizing factual content.
- Fixed-scaffold baseline: compare against an unchanged outer optimizer or harness.
- Resource-dose baseline: equalize extra compute, search, memory, and evaluator calls.
- Protected evaluation: use tests the improving system cannot modify or inspect directly.
Recursive leverage
Conceptually, recursive leverage is the causal effect of inheriting X on a later improvement transition. The quantity can be estimated for different mutable surfaces and different stages of improvement. A matrix can ask, for example, whether a memory change improves later diagnosis, whether a harness change improves later selection, or whether an evaluator change improves later validation.
This matters because recursive development may be uneven. A system could become better at generating modifications while becoming worse at evaluating them. It could improve speed while degrading rollback. A single scalar “self-improvement score” would hide those tradeoffs.
Evidence ladder
Self-change
The system can modify a declared component.
Inheritance
A modification persists and affects later behavior.
Recursive leverage
Inheritance causally improves a later improvement transition.
Falsification
The hypothesis should be weakened or rejected if adequately powered studies repeatedly find that recursive-leverage estimates disappear when compute, information, search, storage, and evaluator access are equalized; if fixed optimizers match the developmental architecture; if stored history can be deleted without changing later improvement; or if apparent gains vanish under sequestered evaluation.
Negative recursive leverage is also meaningful. A retained change might make the system worse at later improvement by corrupting evaluation, overfitting the search process, increasing developmental debt, or degrading rollback.