# 4H MLP Robust Retraining - 2026-06-18

## Context

This run followed the initial 4H retraining attempt documented in `docs/mlp_4h_retrain_2026_06_18.md`. That attempt showed fold-Calmar-only objectives could improve parts of the 4H surface but did not address drawdown, P&L/DD, OOS coverage, or threshold fragility directly.

The canonical MLP inputs are now the normalized files:

```text
data/COINBASE_BTCUSD-{4H,6H,8H,12H,1D}.csv
```

Do not use `data/mlp/MLPScores_*.csv` or `data/mlp/COINBASE_BTCUSD*.csv` as training input. Treat `data/mlp/` exports as TradingView parity/trade-export material only.

`config.TRAIN_END` remained `2026-02-28`, so `2026-03-01` through the latest canonical data remains a true holdout window.

## Tooling Change

Added an explicit robust training objective:

```bash
--fold-objective robust
```

The legacy objectives remain unchanged:

- `mean`
- `min`
- `mean_min`

The robust objective still trains only on the in-sample WFO folds, but subtracts penalties for:

- fold drawdown worse than `50%`,
- fold `P&L/DD Ratio == 0`,
- low fold trade coverage,
- negative fold Calmar,
- threshold fragility at `0.05` and `1.0` score margins.

The artifact records `phase2.robust_diagnostics` so the penalty mix is auditable.

## Serious 4H Run

Command:

```bash
.venv/bin/python tools/train_mlp.py \
  --data data/COINBASE_BTCUSD-4H.csv \
  --hidden 8 4 \
  --phase1-epochs 200 \
  --es-generations 300 \
  --es-popsize 24 \
  --es-sigma 0.04 \
  --l2 0.01 \
  --fold-objective robust \
  --seed 303 \
  --out strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_4H_robust_8x4_seed303.json
```

Environment:

- Unsandboxed training saw Metal and used `phase1: device=mps`.
- Artifact data input: `data/COINBASE_BTCUSD-4H.csv`
- Artifact data SHA-256: `59de4ddbce4ac0b4afd6b249c418b51485426670bfc912548f2daf50e2bf52cb`
- Training window in artifact: `2017-01-01` through `2026-02-28`

Training result:

- Fold objective: `robust`
- Best fitness: `-0.0830951387`
- Fold Calmars: `[0.6026, 0.0861, 0.1806, 1.1188, 1.0624, 0.0000]`
- Mean fold Calmar: `0.5084`
- Min fold Calmar: `0.0000`
- Base fold score: `0.3559`
- Robust diagnostics:
  - valid folds: `5`
  - drawdown penalty: `0.0000`
  - P&L/DD penalty: `0.1667`
  - trade penalty: `0.1000`
  - negative-fold penalty: `0.1667`
  - fragility penalty: `0.0640`
  - fragile bars at `0.05`: `1`
  - fragile bars at `1.0`: `7`
  - fold trade target: `15`
- Recommended thresholds: entry `3.73`, exit `16.43`, confirmation `-28.14`, trailing stop `8.32`

## Sweep Evaluation

The final comparison used a larger sweep because the first 5k pass got close to the drawdown cap.

Command:

```bash
.venv/bin/python tools/run_mlp_deep_sweep.py \
  --timeframes 4H \
  --samples 10000 \
  --top-k 3000 \
  --extra-weights strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_4H_robust_8x4_seed303.json
```

Summary:

- Evaluated candidates: `9,964`
- Sweep file: `results/sweeps/optimization_sweep_strategy_mlp_scores_COINBASE_BTCUSD_4H.csv`
- Summary file: `results/reports/mlp_deep_sweep_summary.csv`
- Promoted: `False`
- Promotable: `False`

Best overall robust row:

| Source | Entry | Exit | Conf | Trail | Trades | P&L % | Max DD % | Calmar | P&L/DD | Sub1 Calmar | Sub2 Calmar | WFO Min | WFO Neg | OOS Trades | OOS P&L % | OOS Sortino | Fragile 0.05 | Fragile 1.0 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `local_2` | `30` | `60` | `-50` | `10` | `116` | `1136.07` | `-39.29` | `0.1328` | `28.91` | `0.0003` | `0.2263` | `-0.0497` | `1` | `9` | `-6.48` | `-0.3590` | `1` | `13` |

Best low-fragility positive-P&L/DD robust row:

| Source | Entry | Exit | Conf | Trail | Trades | P&L % | Max DD % | Calmar | P&L/DD | Sub1 Calmar | Sub2 Calmar | WFO Min | WFO Neg | OOS Trades | OOS P&L % | OOS Sortino | Fragile 0.05 | Fragile 1.0 |
| --- | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: | ---: |
| `broad` | `-265` | `-130` | `130` | `10` | `99` | `171.50` | `-38.34` | `0.0532` | `4.47` | `0.0044` | `0.0929` | `0.0643` | `0` | `6` | `-4.71` | `-0.5653` | `0` | `4` |

## Decision

Do not promote the robust 4H candidate.

The robust objective made real progress:

- reduced drawdown below the P&L/DD cap for the best row,
- found threshold sets with `Fragile_0.05=0`,
- produced several rows with `WFO_Neg_Folds=0`.

It still does not clear the full standard:

- the best row has `Fragile_0.05=1`,
- the best row has one negative WFO fold,
- holdout performance from `2026-03-01` onward is negative,
- the low-fragility row has lower Calmar than the current winner and weak total P&L.

The canonical 4H artifact remains unchanged:

```text
strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_4H.json
```

No Pine preset regeneration is needed.

## Next Experiment

The robust objective is a better direction than fold-Calmar-only retraining, but the 4H holdout still rejects the candidate. The next 4H run should keep the robust objective and test either:

- a slightly larger architecture, such as `16x8`, with the same robust penalties, or
- a lower target-return horizon, such as `--target-k 6`, to make 4H signals less dependent on long high-drawdown holds.

Keep `TRAIN_END=2026-02-28` until a candidate passes holdout validation.