# Spec: Optuna Upgrade — Smarter Parameter Search

**Status:** Planned
**File:** `docs/strategyActivation/spec_optuna_upgrade.md`

---

## What this does for us (layman's version)

Right now the optimizer works like a blindfolded person throwing darts at a dartboard the size of a football field. Each throw is completely random — the previous throws teach it nothing. We throw 24 million darts per iteration, and the best 5,000 hits go to a second round of careful checking. This works, but it wastes most of the budget exploring regions of the space that are obviously bad.

Optuna replaces the blindfold with a probabilistic map. After each batch of evaluations it builds an internal model of "which parameter neighbourhoods tend to produce good results." Subsequent batches concentrate sampling in those neighbourhoods while still occasionally exploring elsewhere (to avoid getting stuck). The result: the same compute budget finds meaningfully better parameter sets, or equivalently, finds comparable results in much less time.

**Concrete analogy:** if the optimizer discovers that `i_w_us10y ≈ +60` tends to be good, random search keeps wasting shots on `i_w_us10y = -80`. Optuna shifts its distribution toward `+60` while still sampling some negatives to stay honest. Over 8 iterations this compounding effect is substantial.

**Why 5–10× sample efficiency claim?** Optuna's TPE (Tree-structured Parzen Estimator) is well-benchmarked on high-dimensional spaces similar to ours (~30 params). In practice 5× is conservative for spaces with clear structure; our sign-stability data already shows several params have strong preferred directions, exactly the structure TPE exploits.

---

## Current architecture

```
optimize_strategy.py
  └─ random_search():
       1. Sample N params uniformly at random (numpy)
       2. Upload to GPU → evaluate all N in one kernel
       3. Return top-5000 by GPU score
       4. CPU-verify top-5000 (parallel workers)
       5. Write sweep CSV + update winner
```

Each iteration is independent. The optimizer has no memory of previous iterations.

---

## The key challenge: batch GPU vs sequential Optuna

Optuna's default mode is **sequential** — suggest one trial, evaluate it, observe the result, suggest the next. This is fundamentally incompatible with our GPU batch path, which evaluates millions of param sets in a single kernel call.

The solution is Optuna's **ask/tell API**: instead of letting Optuna run trials itself, we ask it to *suggest* a batch of param sets, evaluate the batch ourselves (on GPU), then *tell* Optuna the results. Optuna updates its internal model and we ask for the next batch.

```
Proposed flow (per iteration):
  study.ask(n_trials=batch_size)   ← Optuna suggests params
       ↓
  GPU evaluates all n_trials
       ↓
  study.tell(trial, value)         ← Optuna learns from results
       ↓
  ... repeat next iteration ...
```

The batch size is the key tuning parameter. Too small (e.g. 1,000) and we under-utilise the GPU. Too large (e.g. 24M) and Optuna gets one data point per iteration and learns slowly — barely better than random. A sweet spot of **500K–2M samples per batch** lets the GPU run for ~15–30 seconds per batch and gives Optuna 8–16 learning rounds within a 2-hour budget.

---

## Proposed architecture

```
optimize_strategy.py (new)
  └─ optuna_search():
       1. Create/load Optuna study (SQLite journal for persistence)
       2. For each iteration:
            a. study.ask(n_trials=BATCH_SIZE) → param_sets
            b. GPU evaluates param_sets → scores
            c. study.tell() for each trial with its score
            d. CPU-verify top-K from this batch
            e. Write sweep CSV entry + update winner if improved
       3. On completion: dump study summary (importance, best trial)
```

**Persistence bonus:** Optuna stores its trial history in a SQLite journal. This means a run interrupted mid-way can be *resumed* — the study picks up from where it left off rather than starting over. Currently an interrupted run loses all progress.

**Importance analysis bonus:** After a run, `optuna.importance.get_param_importances(study)` gives a ranked list of which params drove objective improvement most. This is a free replacement for the separate SHAP analysis tool.

---

## What stays the same

- `params_strategy_activation_scores_*.json` format — no change
- Sweep CSV format — no change (top-K results per iteration, same columns)
- `sweep_database.db` schema — no change
- GPU kernel — no change
- CPU verification step — no change
- Winner file format — no change
- `auto_optimize_loop.py` interface — no change (it calls `optimize_strategy.py` as a subprocess with the same flags)

The change is entirely inside `optimize_strategy.py`'s search loop.

---

## Where the significant effort lies

### 1. Param space translation (medium)
Optuna's `suggest_float(name, low, high, step=...)` and `suggest_int(name, low, high)` replace our current numpy range sampling. The params JSON is read today as `{start, stop, step}` or `{values: [...]}` — both need a translation layer into Optuna suggest calls. Special cases: `values: [0]` (locked params) must map to a constant, not a suggest call.

### 2. Batch ask/tell plumbing (medium)
Optuna's ask/tell API is designed for single trials. Asking for a batch requires calling `study.ask()` N times, collecting the suggested param dicts, stacking them into the GPU matrix, evaluating, then calling `study.tell()` N times with results. The interface is straightforward but requires careful index tracking so each trial gets the right score back.

### 3. GPU matrix construction from Optuna suggestions (medium)
Today we build the GPU input matrix by sampling numpy arrays directly. With Optuna, each suggested trial comes as a dict of float values. We need to stack N dicts into the same matrix format the GPU kernel expects — column order must still match `_prepare_features()`.

### 4. Sampler warm-start from sweep DB (low–medium, high value)
Optuna can be initialised with prior trial history. This means we can seed the study with the ~11,000 rows already in `sweep_database.db`, giving it a head-start on the first run rather than treating every run as cold. The conversion is: read DB rows → create `optuna.trial.FrozenTrial` objects → pass to `study.add_trials()`. This is the highest-ROI piece relative to effort.

### 5. Fallback / regression safety (low)
Keep the current `random_search()` function intact behind a `--search random` flag. The default changes to `--search optuna`. This ensures we can always fall back if Optuna introduces unexpected behaviour.

---

## Validation plan

### Step 1 — Unit parity test
Run both `random_search` and `optuna_search` on the same asset/TF with a fixed random seed for 1 iteration. Confirm:
- Same winner file format produced
- Same sweep CSV columns
- DB rows inserted correctly
- No crashes on locked params (`values: [0]`)

### Step 2 — Efficiency comparison (same budget)
Run `random_search` and `optuna_search` each for 2h on `COINBASE_BTCUSD-1D` (our most data-rich combo). Compare:
- Best P&L/DD Ratio found
- Best IS Sortino found
- OOS Sortino of the resulting winner (run `oos_dashboard.py` after each)

Accept the Optuna upgrade if OOS Sortino is ≥ equal, or IS Sortino is meaningfully better (>10%).

### Step 3 — Resume test
Kill an Optuna run mid-iteration, restart with the same study name. Confirm it resumes from the correct iteration count and the trial history is preserved.

### Step 4 — Full 20-combo run comparison
After step 2 passes, run the full overnight 20-combo batch with Optuna and compare the resulting OOS dashboard against the last random-search overnight run.

---

## Implementation note: biased sampling vs ask/tell

Benchmarking revealed that Optuna ask/tell runs at ~1,300 trials/s in Python — 500× slower than
our GPU throughput (640K samples/s). Full ask/tell would reduce effective sample rate by 7-8×,
requiring an equivalent efficiency gain just to break even.

The implemented approach: **Optuna-Informed Biased Sampling**
- GPU samples at full speed (same throughput as random)
- 50% of samples biased toward param regions of known-good historical results (from sweep DB)
- Biased ranges refreshed every 5M samples using the current run's accumulated top heap
- Optuna used post-run for importance analysis only (MDI evaluator, ~0.5s for 1K trials)
- No ask/tell during the GPU phase — zero Python overhead
- Net result: same sample count, better distribution → genuine efficiency improvement

The `GPU_BATCH_SIZE` was raised from 1M → 4M (VRAM was at 7% utilisation with 1M).

## Punch list

- [x] **Install Optuna** — added `optuna` + `scikit-learn` to `requirements.txt`; installed in `.venv`
- [x] **Add `--search {random,optuna}` flag** to `optimize_strategy.py` (default: `random`)
- [x] **Write `_build_param_space()` + `_compute_biased_ranges()`** — extracts good regions from DB history; handles `{start,stop,step}`, `{values:[v]}` (locked), and `{values:[v1,...]}` (categorical)
- [x] **Write biased GPU sampling loop** — 50% exploit (biased ranges) + 50% explore (full range); refreshes every 5M samples from accumulated top-heap
- [x] **Sampler warm-start** — `_load_db_top_results()` loads top-K for this asset/TF from sweep DB to seed biased ranges
- [x] **Post-run importance dump** — `_write_importance_report()` writes `results/reports/optuna_importance_{ASSET}_{TF}.md` (MDI evaluator, ~0.5s)
- [x] **`results/optuna_studies/` added to `.gitignore`** (directory reserved for future study persistence if needed)
- [x] **Fallback flag wired through `auto_optimize_loop.py`** — `--search` passed through to subprocess
- [x] **Step 1 unit parity test** — both modes produce identical output file formats; sweep CSV, winner, run_best all correct
- [ ] **Step 2 efficiency comparison on BTC 1D** — run both modes for 2h; compare best Calmar found
- [ ] **Step 3 resume test** — N/A for biased sampling (each run is stateless); persistence is via sweep DB
- [ ] **Flip default to `--search optuna`** after step 2 passes
- [ ] **Step 4 full 20-combo comparison run** (see Validation)
- [x] **Update `CLAUDE.md` common commands** with `--search` flag
- [ ] **Remove `random_search()` flag from docs** once Optuna is default and stable

---

## Open questions

**Q: What batch size?**
Start with 500K. If GPU utilisation drops below 50% (visible in progress output), increase to 1M. Monitor via the existing `Rate: Xk samples/s` output.

**Q: Which Optuna sampler?**
`TPESampler` (default) for standard runs. `CmaEsSampler` is worth trying for the final narrowing iterations where the space is already constrained — it's better at fine-grained local search. Could be a future `--sampler` flag.

**Q: Does the warm-start from DB create data leakage?**
The DB contains IS-period results only (we filter to TRAIN_END). No OOS data enters the study. The warm-start is equivalent to telling Optuna "here are some earlier experiments" — no different in principle from running more iterations.

**Q: What about `i_w_mvrv_cont` and other locked params?**
Locked params (`values: [0.0]`) must not be passed to `study.ask()` — they're constants. The translation layer in `_build_param_space()` must detect single-value entries and return a constant, not a suggest call.

---
_Created: 2026-03-18_