# Marathon Run Retrospective
**Run dates:** 2026-03-19 → 2026-03-22
**Duration:** 74.5h (4 cycles × 20 combos × ~1h)
**Result:** 80/80 OK, 0 failures

---

## What We Hoped to Learn

1. **Deeper DB coverage for alts** — Prior to the marathon, ETH/SOL/LINK had zero v3 rows. We expected 4 cycles × ~2K rows/combo = ~8K top-quartile rows per alt, enough to unlock sign stability analysis and cross-asset lock inference.
2. **Optuna value over random** — Cycle 1 (random) builds DB history; cycle 2 (optuna) should exploit it. We expected optuna to converge on better IS params faster than random, as it demonstrated on ETH 6H (100× improvement pre-marathon).
3. **Param lock cascade** — After cycles 1 and 3 (random), `lock_params --apply --cross-asset` would tighten ranges. With enough alt data, cross-asset locks would broadcast BTC's strong sign verdicts (i_w_macd_bullish NEGATIVE, etc.) to sparse combos.
4. **OOS improvement** — More compute → better IS params → at least marginal OOS improvement on the ETH combos that showed positive OOS.

---

## What We Actually Learned

### 1. Regime is still the binding constraint on OOS

After 80h of optimization, the OOS picture is nearly identical to before:

| Verdict | Combos | Notes |
|---|---|---|
| ✅ ACCEPT | ETH 6H (62%), ETH 12H (51%) | Unchanged from pre-marathon |
| ⚠ WEAK | ETH 8H (34%), SOL 4H (27%) | Minor, statistically thin |
| ⚠ FEW TRADES | ETH 4H (+49.9%), ETH 1D (+13.3%), others | Too few trades to trust |
| ❌ POOR / NEGATIVE | All 15 BTC + SOL + LINK combos | 2025 bear market |

BTC's best IS Sortino improved slightly (4.06 on 1D vs 3.87 pre-marathon), but OOS barely moved (−23.5%, from −32.1%). More optimization is not the path forward for OOS improvement on BTC — this is a regime problem. The strategy was trained on 2017–2024 bull cycles; 2025 is sideways/bear.

**Takeaway:** Parameter optimization has reached diminishing returns for OOS performance. The primary levers going forward are signal quality (new indicators) and regime-aware training (Phase 4 dual-weight system).

### 2. Sign stability is now very well-established for BTC

With 5,019 top-quartile v3 BTC rows (all 5 timeframes at DEEP tier), signal direction is high-confidence:

| Signal | Direction | Confidence | Interpretation |
|---|---|---|---|
| `i_w_macd_bullish` | **NEGATIVE (locked)** | HIGH (0% pos) | MACD crossover bullish flag hurts — likely too lagging |
| `i_w_shooting_star` | Likely negative | HIGH (6.6% pos) | Bearish candle adds bearish weight — hurts long entries |
| `i_w_bullish_engulfing` | Likely negative | HIGH (12.4% pos) | Candle patterns are noise at these timeframes |
| `i_w_stoch_bottoming` | Likely negative | HIGH (16.3% pos) | Counter-intuitive: stoch at bottom ≠ good entry |
| `i_w_rsid_osc` | Likely negative | HIGH (16.8% pos) | RSI divergence hurts — see RSI lib investigation in backlog |
| `i_w_bearish_engulfing` | Likely negative | HIGH (16.9% pos) | Makes sense: bearish signal in long strategy |
| `i_w_osc` | Likely negative | HIGH (18.6% pos) | Oscillator reading as long entry filter is harmful |
| `i_w_us10y` | Likely positive (locked) | HIGH (92.6% pos) | Rising 10Y = risk-on / reflation = good BTC entry |
| `i_w_btc_dom` | Likely positive (locked) | HIGH (90.8% pos) | BTC dominance rising = altcoin capital rotating into BTC |
| `i_w_spy` | Likely positive | HIGH (86.8% pos) | Equity strength = broad risk appetite |

Most remaining weights (stoch, macd_pred, m2/m3 signals, macro) show NO consistent sign direction across BTC timeframes — suggesting they are noise-dominant or timeframe-specific.

### 3. A critical optuna bug was discovered

**In both optuna cycles (cycles 2 and 4), all 15 non-BTC combos (ETH/SOL/LINK × all 5 TFs) added exactly 0 DB rows, while BTC added 2,500–8,900 rows per combo.**

Root cause: `strategies/optimize_strategy.py` has `DB_SCHEMA_VER = 2`, while `tools/auto_optimize_loop.py` has `DB_SCHEMA_VER = 3`. When optimize_strategy.py runs optuna biased search, it queries the DB for `WHERE schema_ver=2 AND asset=? AND timeframe=?`. Since ETH/SOL/LINK were only ever run after the schema bump (all their rows are v3), this query returns 0 rows. Optimize_strategy.py falls back to "No DB history" and runs as random. Something in this code path causes the sweep file to either not be written or not be picked up by auto_optimize_loop.py's `verify_top_results()`, so `write_to_sweep_db()` is never called.

**Impact:** Roughly 30 hours of compute (2 cycles × 15 combos × 1h) produced no DB data for alts. The DB for alts was only built by the 2 random cycles. Without this bug, alts would have ~2× the v3 DB depth.

**Fix required before next marathon:** Update `DB_SCHEMA_VER` in `strategies/optimize_strategy.py` from `2` to `3` (matching auto_optimize_loop.py).

### 4. Sweep dashboard has a stale schema filter

`mine_sweep_db.py` (and thus `sweep_dashboard.md`) currently filters all tier/sign analysis by `schema_ver=2`. Since only BTC has v2 rows (20,049 rows), the dashboard correctly shows only BTC for sign stability — but it should be using v3 rows (224,187 rows), which include all 20 combos. Until this is fixed, the cross-asset universality and timeframe stability sections cannot populate for alts, and per-combo lock verdicts won't appear for ETH/SOL/LINK even though they have thousands of v3 rows.

**Fix required:** Change the schema_ver filter in mine_sweep_db.py from `2` to `3`.

### 5. Marathon infrastructure worked perfectly

- **80/80 runs completed**, 0 failures, runtime 74.5h vs 96h budget (finished early due to no LLM narrowing pauses)
- **ntfy notifications** delivered after each combo — very useful for monitoring
- **Git push per cycle** worked; `results/marathon_progress.md` was committed and pushed 4 times
- **lock_params.py** applied 9 locks correctly across cycles 1 and 3
- **DB grew from ~42K → 256K rows** (224K v3, 32K older)

### 6. BTC IS performance improved across cycles

Best IS P&L/DD Ratio improved meaningfully for BTC across cycles (example, BTC 1D):
- Pre-marathon: Sortino 3.87
- After cycle 1: Sortino 3.87 (minor improvement)
- After cycle 3: Sortino 4.06 (+5%)

BTC 6H showed the most movement, cycling between Sortino 1.55 and 1.70 as different param regions were explored. The optimizer is not stuck — it continues to find modestly better IS configurations — but OOS doesn't track IS gains due to regime mismatch.

---

## What Worked

| Item | Notes |
|---|---|
| Marathon orchestration | Zero crashes, clean lock application, git push, ntfy |
| Random search DB building | All 20 combos built substantial v3 history in cycles 1 and 3 |
| BTC lock stability | All 9 locks held stable across both random cycles — high confidence |
| ETH OOS durability | ETH 6H and 12H held ACCEPT rating across all 4 cycles |
| lock_params cross-asset | Correctly skipped due to insufficient alt data — didn't over-apply |

## What Didn't Work

| Item | Root Cause | Fix |
|---|---|---|
| Optuna adds 0 rows for ETH/SOL/LINK | `DB_SCHEMA_VER=2` in optimize_strategy.py, alts have no v2 rows | Update to `DB_SCHEMA_VER=3` |
| Sweep dashboard shows alts as SPARSE | mine_sweep_db.py filters on `schema_ver=2` | Update filter to `schema_ver=3` |
| Cross-asset lock inference didn't fire | Alts stayed SPARSE (v2 count = 0) so threshold not met | Follows from fixing DB schema version |
| OOS improvement | Not a search problem — regime mismatch | Needs Phase 4 or new signals |

---

## DB State After Marathon

| Asset | v3 Rows | Tier |
|---|---|---|
| BTC (all TFs) | 9,599 – 37,581 | 🔵 DEEP |
| ETH (all TFs) | 2,827 – 9,753 | 🔵 DEEP (but v3, not counted by dashboard) |
| SOL (all TFs) | 4,603 – 8,216 | 🔵 DEEP (same) |
| LINK (all TFs) | 2,449 – 8,113 | 🔵 DEEP (same) |

After fixing the schema filter, all 20 combos will show DEEP tier and become eligible for sign stability analysis (threshold: 200 top-quartile rows, i.e. pnl_dd_percentile ≥ 75).

---

## Recommendations for Future Runs

### Fix first (before any next marathon)

1. **`strategies/optimize_strategy.py` line 35:** Change `DB_SCHEMA_VER = 2` → `DB_SCHEMA_VER = 3`
   This fixes optuna biasing for all alts and ensures their future optuna cycles add DB rows.

2. **`mine_sweep_db.py`:** Update schema_ver filter from `2` → `3`
   This enables the sign stability dashboard to use the 224K v3 rows already in the DB.

After these two fixes, run `python3 mine_sweep_db.py` to regenerate the dashboard — it will likely unlock sign stability analysis for ETH/SOL/LINK immediately.

### Run duration guidance

**Next marathon (recommended): 48h, 2 cycles (random → optuna)**
With the bug fixed, optuna will have real v3 history for all combos. The value of a second random cycle after a working optuna cycle is lower than the first. Two cycles × 20 combos × 1h = 40h active + some overhead. This tests whether optuna genuinely improves over random for alts (the original question), and will push all combos comfortably DEEP on v3 data.

**Long-term:** Don't run longer marathons until new signals are added. With current signal set, IS Sortino is converging — BTC 1D has likely reached near-optimal params. Compute is better spent on:
- Adding US 2Y yield, QQQ/SPY ratio (backlog signals)
- Phase 4 regime-switching (train separate bull/bear param sets)

**Calmar vs P&L/DD evaluation:** We now have enough v3 DB data to run the analysis suggested in the backlog — compare whether `calmar_ratio` or `pnl_dd_ratio` as IS objective correlates better with OOS outcomes. Pull from `results/sweep_database.db` and check which IS metric predicts OOS Sortino better across the ETH combos that have positive OOS.

---

## Interesting Observations

- **ETH is the only asset with positive OOS across all timeframes** (at least directionally, even "FEW TRADES" combos like ETH 4H +49.9%). This isn't random — ETH likely has different regime dynamics in 2025 (e.g., range-bound vs BTC's specific downtrend). It suggests the strategy IS working on ETH but 2025 BTC/SOL/LINK were in a regime the training window didn't prepare for.

- **Candle patterns are consistently harmful.** `i_w_bullish_engulfing`, `i_w_bearish_engulfing`, `i_w_bullish_hammer`, `i_w_shooting_star` all show sign stability toward zero or negative. At daily/4H timeframes on crypto, individual candle patterns appear to be noise. Consider zeroing these weights out or locking them to `[0, 0]` — they are consuming search space for no benefit.

- **`i_w_macd_bullish` deserves special attention.** It is the most consistently negative signal in the entire set (0.0% positive across 5,019 top-quartile rows). The MACD-bullish flag marks when MACD crosses above signal. That this is *harmful* to entries suggests the strategy is not a momentum follower — it actually entries better when MACD has NOT recently turned bullish (i.e., before the crowd notices).

- **BTC 8H is stuck at 2 OOS trades.** This is a degenerate edge case — the IS winner has an entry threshold that rarely fires in 2025 data. Could be worth investigating if the threshold is pathologically high.

- **Schema v4 rows exist (11,585 rows, 2026-03-18 12:49–12:55).** These are from a very short run before the marathon started, likely the tail of the alts comparison run. They predate the v3 marathon data and don't affect the analysis, but suggest there was a brief period where DB_SCHEMA_VER was set to 4 in auto_optimize_loop.py. Worth checking if that's still the case or was reverted.

---

_Last updated: 2026-03-22 (marathon retrospective)_