# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

> For the full command reference (all tools, all flags), see [docs/mlp_training_runbook.md](docs/mlp_training_runbook.md).

## Before Any Training or Sweep Run

```bash
python3 tools/sync_params.py     # must report ✅ (all 20 params files in sync with config.py)
python3 tools/check_pine.py      # must exit 0 (no arch mismatch, no hard-limit violations)
```

**Before adding any `request.security()` call to strategy_mlp_scores.pine or any library it imports:**

```bash
python3 tools/count_pine_security_calls.py --strategy strategies/strategy_mlp_scores.pine
# must show ≤40 unique (symbol, tf) pairs — follows library imports, deduplicates properly
# check_pine.py does NOT cover the MLP Pine's library overhead; use this tool instead
```

A pre-commit hook enforces this automatically on any staged `.pine` file.

## MLP Active Commands

```bash
# ── Status & results ──────────────────────────────────────────────────────────
python3 tools/mlp_status.py                                    # winner per TF, arch, Pine sync
python3 tools/mlp_results_table.py                             # IS + OOS metrics (BTC) + saves results/mlp_performance.md
python3 tools/mlp_results_table.py --asset COINBASE_ETHUSD     # always pass --asset for non-BTC

# ── Adaptive training (preferred — detects convergence, switches to bull regime automatically)
python3 tools/run_mlp_adaptive.py --notify                     # all 5 BTC TFs
python3 tools/run_mlp_adaptive.py --tfs 4H 8H --notify
python3 tools/run_mlp_adaptive.py --dry-run                    # preview plan

# ── Manual training orchestrator ─────────────────────────────────────────────
python3 tools/run_mlp_train.py --assets COINBASE_BTCUSD --seeds 3 --sweep --promote --presets --notify
python3 tools/run_mlp_train.py --assets COINBASE_BTCUSD --tfs 6H 8H --seed-start 404 --seeds 3 --sweep --promote
python3 tools/run_mlp_train.py --assets COINBASE_BTCUSD --seeds 3 --regime bull --sweep --promote --notify

# ── Threshold sweep only (weights fixed) ─────────────────────────────────────
python3 tools/run_mlp_deep_sweep.py --asset COINBASE_BTCUSD --timeframes 8H \
  --samples 200000 --wide --promote \
  --extra-weights strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_8H_bb50_bull_seed*.json
# After any --promote or --force-promote: run generate_pine_mlp_presets.py (no auto-presets in deep sweep)

# ── Pine presets ──────────────────────────────────────────────────────────────
python3 tools/generate_pine_mlp_presets.py    # rewrite sentinel block in strategy_mlp_scores.pine
# Then: open strategies/strategy_mlp_scores.pine → Select All → Copy → paste into TradingView.
# mlp_status.py must show ✅ synced for all promoted TFs before exporting from TV.

# ── Parity check (after every TV re-export) ───────────────────────────────────
python3 tools/validate_chart_data.py                           # ALWAYS run first — see Parity Rules below
python3 tools/check_mlp_parity.py --asset COINBASE_BTCUSD --tf 6H   # use --asset/--tf, never --weights
python3 tools/compare_tv_trades.py \
  --tv-trades "data/mlp/MLPScores_COINBASE_BTCUSD_<date>_<tf>.csv" \
  --data "data/mlp/COINBASE_BTCUSD, 240.csv" \
  --params results/winners/optimization_winner_strategy_mlp_scores_COINBASE_BTCUSD_4H.csv
```

## Testing

```bash
python -m pytest tests/ -q              # fast tests only
python -m pytest tests/ -q --run-slow  # includes smoke train (~5 min)
```

## MLP Architecture

- **Shape**: `55→16→8→1` (BTC); alts `50→16→8→1` pending 55-feature retrain. `mlp_status.py` is the source of truth for live arch + winning seed per TF — do not trust prose references here.
- **Activation**: `tanh` closed form: `x≥20 ? 1 : x≤-20 ? -1 : (exp(2x)-1)/(exp(2x)+1)`
- **Score**: `1000 × tanh(W3 · tanh(W2 · tanh(W1·x + b1) + b2) + b3)` — ±1000 scale
- **Hard-locked params**: `i_div_window=1`, `i_m3_momentum_period=1` — hardcoded in both Pine and Python; do not sweep.
- **Feature cols 3/6/10** are now real Glassnode on-chain signals (`totalvol_norm`, `newaddr_norm`, `sendaddr_norm`) as of 2026-06-25. Previously always 0; retrained all BTC TFs after column rename.

## Data Flow

Pine normalises all signals to `[-1, +1]` via `plotchar` and exports them as `_norm` columns in the TV CSV. Python reads these directly — no re-normalisation. This keeps the two codepaths in sync by construction.

**Adding a new signal (all 7 steps required):**
1. Compute normalised value in Pine; export via `plotchar` with `_norm` suffix.
2. Read in `strategy_activation_scores.py` with `get_col('my_signal_norm')`.
3. Add weight `i_w_my_signal` to weighted sum and `max_score` denominator.
4. Append `get('my_signal_norm')` to `_prepare_features()` `cols` list, same order as the JSON key.
5. **Add `"i_w_my_signal"` to `WEIGHT_COLS` in `config.py`** — propagates to sweep DB, SHAP, and schema automatically.
6. **Run `python3 tools/sync_params.py --apply`** — propagates the new key to all 20 active params files.
7. Re-export data CSVs; run `compare_tv_trades.py` to verify delta ≈ 0.

`auto_optimize_loop.py` aborts at startup if steps 5 or 6 are skipped.

## plotchar Slot Budget (near 64-slot limit)

Two categories — confusing them causes severe parity bugs:

| Category | Description | Safe to remove? |
|---|---|---|
| **Score weight** | Feeds into score sum only | Yes, if weight is locked=0 |
| **Condition gate** | Used in entry/exit `if`/`and`/`or` logic | **Never** — weight value is irrelevant |

**Must keep (condition gates):** `stoch_peak_norm` — gates `longExitCondition`. Removing it silences Python's exit crossunder, causing Python to hold positions for weeks/months while TV exits normally.

**Safe to remove** (locked=0, score-sum only): `stoch_bot_norm`, `macd_bullish_norm`, `rsid_norm`

Before removing any `plotchar` line: search the full Pine file for the variable name and confirm it does not appear in any `if`/`and`/`or` condition outside the score sum.

## Param Locks

**Accepted single-value locks** (intentional — do not widen):
- `i_w_stoch_peaking`, `i_w_stoch_bottoming` — condition gates, not score weights
- `i_w_macd_bullish`, `i_w_rsid_osc` — empirically always near-zero
- `i_m3_momentum_period=1`, `i_regime_window=0`

**Must NOT be locked to a single value** (unlock immediately if found):
- `i_use_long_entry_confirmation`, `i_use_long_exit_confirmation` — boolean, must search both True/False
- `i_w_btc_dom`, `i_w_nupl`, `i_w_fed_net_liq` — valid for all assets; sign varies by asset

## TV/Python Parity Rules

**Before diagnosing any trade-list divergence:** re-export the data CSV and trade list from the same TV state before theorising an engine bug. Score parity passing is not evidence the data is current — a stale CSV can be internally consistent with a stale trade list while both diverge from the current Pine state.

**Always run `validate_chart_data.py` after every TV re-export** before trusting any backtest number. It catches truncated/partial exports that silently produce wrong results. Score parity does not catch this.

**Do not hand-pass `--weights` to `check_mlp_parity.py`.** The plain `mlp_weights_{ASSET}_{TF}.json` files are often stale non-winning artifacts. Always use `--asset`/`--tf` to auto-detect the current winner.

**When a sweep winner fails post-deployment:** distinguish "feature broken" from "simulation broken" before acting. Fix the simulation and re-sweep; don't revert the feature. A measurement flaw crowds rankings with flaw-exploiters and shadows genuinely good candidates that were previously hidden behind them.

## Promotion & New Assets

`promotable()` requires the candidate to beat the current winner in both Calmar and P&L/DD, pass WFO gates, and not materially degrade OOS Sortino. For a new asset with no prior winner, `promotable()` always returns `False` — use `--force-promote`.

**After any feature column rename** (e.g. replacing cols 3/6/10 with new signals): all existing winner artifacts expect the old column names and are incompatible with the new CSVs. The adaptive loop reads Calmar from the winner CSV (not by re-running the model), so it uses the old-data baseline and new seeds will never beat it. `run_mlp_adaptive.py` now detects this and resets the baseline to 0 automatically, but if you bypass the loop use `--force-promote` explicitly:

```bash
python3 tools/run_mlp_deep_sweep.py \
  --asset COINBASE_ETHUSD --timeframes 4H 6H 8H 12H 1D \
  --samples 80000 --force-promote \
  --extra-weights strategies/params/mlp/mlp_weights_COINBASE_ETHUSD_*_bb50_seed*.json
```

## Phase 2 Upgrade (adding features to Pine forward pass)

Run `/mlp-phase2` for the interactive checklist. Pre-conditions: all TFs must share the same arch (`mlp_status.py`) and `check_pine.py` must exit 0 before starting.

## Assets & Timeframes

4 assets × 5 TFs = 20 combos:
- Assets: `COINBASE_BTCUSD`, `COINBASE_ETHUSD`, `BINANCE_SOLUSD`, `BINANCE_LINKUSD`
- Timeframes: `4H`, `6H`, `8H`, `12H`, `1D`
- IS window: `SCORE_START` (2017-12-01) → `TRAIN_END` (2026-02-28); OOS: `OOS_START` (2026-03-01+). **Source of truth: `config.py`.**

## Activation Strategy (legacy)

For the activation-scores strategy (auto_optimize_loop, run_all_crypto, run_btc, lock_params, tighten_params, mine_sweep_db), see [docs/activation_strategy_commands.md](docs/activation_strategy_commands.md).

```bash
# Notify (Claude uses this in scripts; also usable directly)
python3 tools/ntfy.py "Message" --title "Title" --priority high
```