# Comparable MLP incumbent evidence — migration, September 21, 2026

The authoritative comparison is now a reproducible replay of the incumbent and candidates under one captured data/execution/accounting contract. Historical winner CSVs remain unchanged provenance. Their metrics never become independent numerical maxima against a current replay. An older export is a legitimate common historical comparison input; elapsed time since export does not by itself make the comparison unfair.

## Affected paths and authority

`run_mlp_deep_sweep.evaluate` produces fingerprinted evidence from immutable captured data and artifact bytes. Ranking and promotion consume those rows; comparison context and successful incumbent evidence are prerequisites. `run_mlp_adaptive.read_winner_evidence` calls the same `replay_incumbent` evaluator. Adaptive progress uses comparable before/after rows and rejects changed context, a failed replay, or an unreproduced baseline. Historical best scores no longer set its starting floor. The training orchestrator `run_mlp_train.sweep_one` delegates comparisons to the sweep; it has no separate incumbent score floor.

Historical experimental runners such as `run_mlp_score_structure_experiment` and `run_mlp_state_policy_experiment` load winner parameters and replay performance. Their reports are historical diagnostics, not inputs to production promotion/adaptive decisions. Historical search and training objectives have not become independent holdouts merely because the metric comparison is corrected.

The fingerprint covers the common data content, ordered feature schema, fee and execution contract, calendar bounds and folds, coverage rules, and relevant evaluator/accounting/feature/config source hashes. Artifact content and parameter identity distinguish evaluated configurations and caches within the common context. Cached frames/metrics cannot be reused across changed inputs. Incumbent failure or incomplete coverage closes the comparison; it does not install a zero floor. A current successful replay establishes today's historical comparison baseline without explaining the older result's discrepancy.

## Preserved requirements and explicit semantics

The promotion conditions remain positive P&L/DD, the existing fragility constraint, positive subperiods, qualifying WFO requirements, matching or beating incumbent Calmar and P&L/DD, and the OOS Sortino degradation limit of 1.0 with its existing trade-count and bear-regime exemptions. The existing drawdown/trade gates remain, including the P&L/DD eligibility band of 0.1–40% drawdown. No condition was tuned to the archived candidates.

Semantic changes are explicit:

- Stored Calmar, P&L/DD, OOS Sortino and OOS trade counts no longer override a current comparable replay. Thus the existing low-trade OOS exemption uses replayed trade counts on both sides, not historical counts from another state.
- Complete IS endpoints and at least two retained OOS bars are required. Invalid/missing model inputs, nonfinite parameters or outputs, empty incumbent files, or nonreproducing incumbent results stop comparison. The force flag remains available only for a genuinely absent incumbent.
- Artifact schemas must match the incumbent's ordered schema within a comparison; valid older 50-feature asset schemas can still define their own common context. The final unconfirmed export candle is excluded for everyone. Internal time gaps remain visible limitations under the declared coverage contract.
- Additional threshold tuning uses the same artifact-neutral seeded threshold vectors and trial count for every distinct artifact path, with one frozen configuration retained per model. The old incumbent-centered local proposal distribution is replaced to remove unequal search treatment. Ranking still uses the existing IS/subperiod/WFO and previously inspected OOS fields; this is not newly independent validation.
- Adaptive stopping uses the same replay evaluator, locks both context and incumbent configuration identity, and aborts failed sweeps/context drift rather than counting them as convergence. Valid small improvements update the replay baseline even below the unchanged convergence threshold.

Measurement and eligibility are separate: every WFO period plus IS, subperiods and OOS is retained with actual performance, including inactive/low-trade periods and carry-in holdings. Qualifying-fold aggregation retains the existing selection rule; an omitted low-trade period is visible and is not presented as a successful robustness test. The unconfirmed final export row is excluded symmetrically, and required features, temporal coverage and finite prices are checked. These are declared measurement rules, not a recency requirement.

## Bounded archived comparison

The migration runner evaluates the incumbent plus grouped, random-sparse and hybrid BTC6H seeds 9090, 9191 and 9292 using saved/recommended configurations. **Every model receives zero additional threshold trials.** Historical budgets remain unequal: the earlier audit counted 66 dense bb50 and 36 bull surviving BTC6H artifacts, versus three seeds per alternative family. Those counts are not complete search ledgers.

The report records two old decisions: the archived old evaluator/promotion path, and the old promotion policy applied to the same corrected replay rows. The latter isolates removal of incompatible metric maxima from changes in snapshot/coverage measurement. The archived evaluator imports the current shared accounting code; it represents the pre-migration comparison path, not the pre-accounting-audit implementation.

The final verified replay found no promotion-decision or ranking changes among the ten models. The incumbent remains first. All nine candidates remain ineligible for promotion; the JSON names every failed preserved condition. Reference conditions change because stored P&L/DD 413.6713 no longer overrides the comparable incumbent value. In this replay the incumbent's 50.4976% drawdown invokes the existing eligibility gate, so the comparison P&L/DD is zero while raw measured return/drawdown remain visible. Calmar is 1.004489 and return is 2,846.943741%, versus historical saved Calmar 0.482796 and return 13,839.525309%. The earlier audit's unresolved historical return discrepancy remains unresolved; calendar annualization alone does not explain it.

The actual adaptive evaluator reproduced the same fingerprint and incumbent Calmar; unchanged-incumbent progress was exactly zero. All 924 protected artifact hashes matched before and after. No model was trained/promoted, no historical winner metrics were rewritten, and no Pine presets or deployment were changed.

## Evidence and reproduction

The canonical evidence is `results/mlp_comparison_migration_2026_09_21/verified/`. Its recorded source hashes were checked against the final working tree. Earlier `replay_initial/` and `replay_final/` runs are retained as preliminary evidence and are not authoritative.

Final verification: **228 tests passed, 2 skipped**, with three existing warnings (one unregistered slow marker and two intentional older-schema fixtures). The log is `results/mlp_comparison_migration_2026_09_21/tests_full.log`. `git diff --check` passes; this repository has no configured lint/typecheck command. `worker_smoke.py` in the evidence root independently compared identical rows across sequential evaluation, two process workers and two subprocess shards; all matched exactly, with all 924 protected hashes unchanged. No additional threshold tuning occurred in that check.

The simplification pass used the ce-simplify-code reuse, quality and efficiency rubrics inline after the harness rejected another review-agent launch at its thread limit. No additional changes were needed after removal of obsolete candidate-generation code and caching immutable snapshot/artifact hashes; consolidating small failure checks was declined to preserve clear failure boundaries. Independent correctness review is recorded separately in `results/mlp_comparison_migration_2026_09_21/independent_review.md`. Each run refuses a nonempty output directory and records prerequisite logs, frozen configurations, source snapshots/hashes, comparison manifest, complete old/corrected rows, every decision, changed reference conditions, failed preserved conditions, and protected-file verification.

```bash
.venv/bin/python tools/verify_mlp_comparison_migration.py --output /tmp/mlp_comparison_migration_reproduction
.venv/bin/python -m pytest tests/test_mlp_comparison.py tests/test_mlp_adaptive_comparison.py -q
.venv/bin/python -m pytest tests/ -q
```

The runner itself executes `sync_params.py` and `check_pine.py` before any evaluation. It never calls candidate generation, training, promotion, preset generation or deployment. For reproduction use the recorded source version and the unchanged hashed input artifacts; later source changes legitimately produce a different fingerprint.

| Requirement | Authoritative evidence |
|---|---|
| Stored incumbent metrics and adaptive history traced | Sweep `promotable`/ranking and adaptive `read_winner_evidence`/`comparison_delta`; orchestrator delegates sweep |
| Common immutable data, schema, execution, fees, calendar, accounting, coverage | Captured comparison manifest plus source and artifact hashes |
| Cache compatibility | Focused regression tests for identical context reuse and changed data/accounting/artifact invalidation |
| Historical discrepancies retained, failures closed | Protected hashes, complete old/current rows, and failed/incomplete incumbent tests |
| Ranking/promotion/adaptive baseline parity | Actual archived adaptive fingerprint/Calmar identity and zero-delta assertion, plus focused tests |
| All periods visible and measured independently of eligibility | `Period_Measurements` in every corrected replay row; carry-in and low-trade tests |
| Equal extra tuning and honest prior budget | Frozen ten configurations, zero trials per model, historical archive-count disclosure |
| Old versus corrected decisions and every difference | Per-model decisions, reference differences, replay-metric differences and preserved-condition failures |
| No weakened hurdle | Focused hurdle regressions and independent review receipt recorded at final verification |
| No promotion, broad retraining, historical overwrite, preset change or deployment | Runner has no mutation entry points; 924 protected hashes match before/after |

After verification, ordinary sweep and adaptive commands use the new evaluator directly; no winner rewrite or score migration command is required. Existing historical report files remain historical records, never compatible cached comparison evidence merely because their filenames match. Independent holdout evidence, historical feature publication vintages and fresh TradingView parity remain separate limitations; this migration makes the shared historical comparison coherent, not a deployment recommendation.