# Tools

This directory contains various utility scripts for analyzing, debugging, and managing the trading strategies.

## Trade Comparison

*   `compare_tv_trades.py`: Loads a TradingView-exported trade list CSV and compares it side-by-side against the Python backtest signals. Reports: match rate, per-trade P&L diff, margin call trades (TV-only), commission drag estimate, and aggregate P&L totals. Default inputs: `results/2025.10.25_ActivationScores_*.csv`, `results/TV_Export.csv`, `results/optimization_winner_activation_scores.csv`. Use `--date-tolerance N` to adjust the entry-date matching window.

## Analysis & Debugging

*   `analyze_diffs.py`: Analyzes a `comparison_debug.csv` file (generated by `compare_strategies.py`) and prints the average and max differences between Python and TradingView versions of several indicators.

*   `analyze_tv_export.py`: Analyzes a `TV_Export.csv` file and compares the `DB_` prefixed columns with their non-`DB_` counterparts to find discrepancies within the TradingView export itself.

*   `check_numba_gpu.py`: Checks if Numba can access the GPU and sets the appropriate environment variables if running in WSL.

*   `compare_strategies.py`: Compares the output of a Python strategy with a TradingView export CSV. It calculates the difference for each debug column and saves the detailed results to `comparison_debug.csv`.

*   `debug_gpu_cpu_discrepancy.py`: A script for diagnosing discrepancies between the CPU and GPU execution paths of the backtest. It uses a fixed set of parameters to run both versions and compares the results.

*   `debug_tv_discrepancy.py`: Another script for comparing a Python-generated CSV with a TradingView export. It's similar to `compare_strategies.py` but with a different column matching approach.

*   `diagnose_components.py`: Compares the debug columns (`DB_`) from a TradingView export and a Python strategy output. It calculates correlation and mean absolute error to identify discrepancies.

*   `diagnose_missing_data.py`: Checks if all the required columns for the Python strategy are present in the `TV_Export.csv` file.

## Data Management

*   `clean_data.py`: Truncates CSV files to a specific start date to align data for consistent warmup periods.

*   `fetch_macro_data.py`: Merges M2 and M3 money supply data from local CSV files into the main BTC price CSV.

## Strategy Optimization & Execution

*  `lock_params.py`: Performs locking of params to narrower range, if the data supports it. Can also be used for a dry run to first check which params would be locked `python3 tools/lock_params.py 2>&1 | grep -v "^$"`

*  `sync_params.py`: Ensures all 20 active params files contain every key listed in `config.WEIGHT_COLS`. Run this after adding a new signal to `WEIGHT_COLS` — it adds missing keys with a wide default range so the optimizer doesn't silently treat them as zero. Also run after `reset_results.py --reset-params` to verify all files are in sync. **Does not modify existing keys or reset locked/narrowed ranges** — use `reset_results.py --reset-params` for that.

  ```bash
  python3 tools/sync_params.py          # dry run — show what would be added
  python3 tools/sync_params.py --apply  # write missing keys to all params files
  ```

  The auto-optimize loop calls this as a preflight check at startup and aborts if any file is out of sync.

*   `walk_forward_runner.py`: Performs a walk-forward analysis on a strategy, which is a more advanced and realistic backtesting technique.

### Out-of-Sample Performance Dashboard (`tools/oos_dashboard.py`)

Reads each winner CSV from `results/`, runs the params on the **full** data (2015→present), and produces a side-by-side comparison of in-sample (2019–2024) vs out-of-sample (2025→latest) performance at `results/oos_dashboard.md`. Also auto-runs at the end of every `auto_optimize_loop.py` run.

**Usage:**

```bash
python3 tools/oos_dashboard.py                              # all assets/TFs
python3 tools/oos_dashboard.py --asset COINBASE_BTCUSD      # BTC only
python3 tools/oos_dashboard.py --asset COINBASE_BTCUSD --timeframe 8H
```

**Verdict tiers:**

| Verdict | Meaning |
|---|---|
| ✅ EXCELLENT | OOS Sortino ≥ 80% of IS Sortino |
| ✅ ACCEPT | OOS Sortino ≥ 50% of IS Sortino |
| ⚠ WEAK | OOS Sortino ≥ 25% of IS Sortino |
| ⚠ FEW TRADES | < 10 OOS trades — Sortino unreliable, shows P&L direction |
| ❌ NEGATIVE OOS | Negative OOS Sortino with sufficient trades |
| ❌ POOR | OOS Sortino < 25% of IS Sortino |

Note: the strategies make ~4–11 trades/year, so 14.5 months of OOS data yields only 5–15 OOS trades — too few for Sortino to be statistically reliable. The 8H timeframe (15 OOS trades) is currently the most informative.

---

### Sweep Database Dashboard (`mine_sweep_db.py`, root directory)

Reads `results/sweep_database.db` — the persistent SQLite database populated by every `auto_optimize_loop.py` run — and writes `results/sweep_dashboard.md` (plus stdout). The dashboard is designed to be read by Claude via the `Read` tool for analysis.

**Usage:**

```bash
# All combos in the DB
python3 mine_sweep_db.py

# Filtered to one asset/timeframe
python3 mine_sweep_db.py --asset COINBASE_BTCUSD --timeframe 1D

# Auto-run at end of an optimization loop
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-1D.csv --hours 0.5 --dashboard
```

**Automatic step adjustment:**

When a `✂ Tighten` flag is shown in the Range Tightening table, the dashboard also computes whether the current `step` size still gives adequate grid resolution over the narrowed range. It targets ≥ 10 grid points for weight params (step ≤ 5) and ≥ 8 for threshold params (step > 5). If the tightened range would fall below those minimums, the suggested action includes `step→N` showing the reduced step to use. Apply this when editing the params JSON alongside the new `start`/`stop` values.

**Dashboard sections:**

| Section | Minimum data | What it shows |
|---|---|---|
| Data Inventory | Any | Rows, runs, last updated, tier per combo. Progress estimate to next tier. |
| Sign Stability | 50 rows | % of top-quartile results with a positive value per weight. Identifies params with a strong directional bias that could be locked. |
| Cross-Asset Universality | 200 rows × 2 assets | Mean weight value per asset on its best-data timeframe. Low spread + same sign = universal signal, candidate for locking. |
| Timeframe Stability | 200 rows × 2 TFs | How much each weight's mean drifts from 4H → 1D per asset. Stable params could be shared in a multi-timeframe strategy. |
| Range Tightening | 200 rows | q5–q95 of top-quartile rows vs. current JSON range. Flags where 90% of mass fits in <70% of the current range. |
| Degenerate Results | Any | Entry ≈ exit threshold (within 10 units) or >500 trades. High % = threshold ranges need constraining. |
| Action Summary | Any | Distilled checklist: HIGH CONFIDENCE locks, MEDIUM CONFIDENCE range narrowing, NOT YET ACTIONABLE sparse combos. |

**Data sufficiency tiers:**

| Tier | Rows | Reliable analyses |
|---|---|---|
| 🔴 SPARSE | < 50 | None — directional signals are noise. |
| 🟡 EMERGING | 50–200 | Sign stability hypotheses only. |
| 🟢 USABLE | 200–500 | Sign stability and range tightening are actionable. |
| 🟢🟢 RICH | 500+ | Cross-asset universality, timeframe stability, and multi-modal clustering are reliable. |

The database accumulates all CPU-verified results with `P&L/DD Ratio > 0` across every run. Unlike winner-only files, it stores the full distribution so you can distinguish "this param is almost always negative" (strong signal) from "this param happened to be negative in the 3 results I have" (noise).

**Resetting the DB:**

```bash
# Delete one combo's rows (e.g., after re-exporting TV data with different columns)
python3 tools/auto_optimize_loop.py --reset-db COINBASE_BTCUSD 4H

# Wipe entire DB
python3 tools/auto_optimize_loop.py --reset-db
```

---

### Auto-Optimization Loop (`tools/auto_optimize_loop.py`)

The main optimization driver. Runs iterative GPU random (or Optuna-biased) search and accumulates all results in `results/sweep_database.db`.

**Arguments:**

| Flag | Default | Description |
|---|---|---|
| `--data` | required | Path to OHLCV data CSV. Asset and timeframe are extracted from the filename. |
| `--hours` | required | Total runtime budget in hours. Divided evenly across iterations (~15 min each). |
| `--search` | `random` | `random` — pure GPU random search. `optuna` — biased sampling from DB history (requires prior DB rows to be effective). |
| `--regime` | `all` | Pre-filter IS bars by MVRV regime: `all` / `bull` / `bear` / `sideways`. |
| `--params` | auto | Override the params JSON file (default derived from `--data` filename). |
| `--no-dashboard` | off | Skip regenerating the sweep dashboard after each iteration. |
| `--reset-db` | — | Delete DB rows then exit. No args = wipe all. Two args (ASSET TF) = wipe that combo. |

**Example commands:**

```bash
# Random search — safe for any run length, good starting point
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-4H.csv --hours 2

# Optuna-biased search — uses DB history to bias 50% of samples toward good regions
# Recommended after ≥1 random run has populated the DB for this asset/TF
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-4H.csv --hours 4 --search optuna

# Optuna on 1D with LLM-guided narrowing (only ≥6h runs)
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-1D.csv --hours 8 --search optuna --llm

# Bull-regime only (filter IS bars to MVRV bull phase)
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-4H.csv --hours 4 --regime bull

# Wipe sweep DB rows for one combo and restart fresh
python3 tools/auto_optimize_loop.py --reset-db COINBASE_BTCUSD 4H
python3 tools/auto_optimize_loop.py --data data/COINBASE_BTCUSD-4H.csv --hours 4
```

**When to use Optuna vs random:**

- **First run on a combo**: always use `random`. Optuna needs DB history to be useful.
- **After ≥1 random run**: switch to `--search optuna`. It biases 50% of samples toward regions that previously scored well, and writes a param importance report to `results/reports/`.
- **Alternating**: the marathon runner (`run_marathon.py`) automatically alternates random → optuna → random between cycles.

**Key rule: only pass `--llm` on runs of 6+ hours.**

Short runs sample an infinitesimal fraction of the search space. The "winner" found is largely random noise. If the LLM narrows the params around that noise, it permanently corrupts the asset-specific params file — locking out large regions of the space for all future runs.

**When to re-export TV data:**

At startup, the loop checks that every `i_w_*` weight param in the params file has a matching `_norm` column in the data CSV. If any are missing, it warns you and prompts to continue or quit. The normal action is to quit, re-export the TV chart data to `data/`, and restart.

## Batch Optimization Runners

*   `run_all_crypto.py` (root): Runs `auto_optimize_loop.py` on all 4 crypto assets × 5 timeframes (20 combos) sequentially.

    ```bash
    python3 run_all_crypto.py --hours 0.5       # 0.5h per combo = ~10h total
    python3 run_all_crypto.py                   # default iterations per combo
    python3 run_all_crypto.py --skip BINANCE_LINKUSD-1D
    ```

*   `run_btc.py` (root): Same as above but restricted to `COINBASE_BTCUSD` across all 5 timeframes. `--hours` is a **total budget** divided evenly across timeframes. Supports an optional `--tfs` filter to run a subset.

    ```bash
    python3 run_btc.py --hours 10               # 10h total ÷ 5 TFs = 2h each
    python3 run_btc.py --hours 8 --tfs 6H 8H 12H 1D  # 8h ÷ 4 TFs = 2h each
    python3 run_btc.py                          # default iterations, all TFs
    ```

---

### DB Query Tool (`tools/query_db.py`)

Query `results/sweep_database.db` and surface top-N rows sorted by any metric. The primary use case is finding parameter sets that weren't the IS winner but have strong performance characteristics — for OOS analysis, cross-asset comparison, or production candidacy.

**Usage:**

```bash
# Top 20 by composite_score for BTC 4H (default)
python3 tools/query_db.py --asset COINBASE_BTCUSD --tf 4H

# Top 10 by Calmar, only rows with ≥20 trades
python3 tools/query_db.py --asset COINBASE_BTCUSD --tf 1D --sort calmar_ratio --top 10 --min-trades 20

# Only subperiod-consistent rows (Calmar > 0 in both IS halves)
python3 tools/query_db.py --asset COINBASE_BTCUSD --tf 4H --subperiod-only --top 50

# Export full param rows (all 80 columns) to CSV for further analysis
python3 tools/query_db.py --asset COINBASE_BTCUSD --tf 4H --top 100 --out results/candidates_btc4h.csv

# Show weight columns inline in the printed table
python3 tools/query_db.py --asset COINBASE_BTCUSD --tf 4H --show-weights --top 5

# List all available column names
python3 tools/query_db.py --list-columns
```

**Key flags:**

| Flag | Default | Description |
|---|---|---|
| `--asset` | all | Filter by asset (e.g. `COINBASE_BTCUSD`) |
| `--tf` | all | Filter by timeframe (e.g. `4H`) |
| `--sort` | `composite_score` | Column to sort by (descending) |
| `--top` | 20 | Number of rows to return |
| `--min-trades` | — | Minimum `total_trades` filter |
| `--subperiod-only` | off | Only rows with `subperiod_consistent=1` |
| `--out` | — | Write full rows (all columns) to CSV |
| `--show-weights` | off | Include `i_w_*` columns in printed table |
| `--list-columns` | — | Print all column names and exit |

**Research task:** Run `query_db.py` across all 20 combos to explore whether strong IS performers share consistent weight sign patterns across assets/timeframes. Key questions:
- Are there param sets with lower IS composite but strong Calmar/Sortino across multiple assets? (Universal signal candidates for locking.)
- Do top-10 rows per combo cluster in the weight space, or are they scattered? (Indicates whether a single weight set could serve all TFs for one asset.)
- Can non-winner rows with positive OOS P&L inform better starting ranges for the next optimization cycle?

---

## Utility

*   `export_params_to_tv.py`: Exports the winning parameters from an optimization run to a Pine Script input format, making it easy to update the strategy in TradingView.

*   `gpush.py`: A git helper script that uses an AI model (Ollama or Gemini) to generate a commit message and then pushes the changes to GitHub.

*   `reset_results.py`: Deletes winner CSVs, cheatsheets, Pine snippets, and the sweep DB. Run before any change that invalidates prior optimization results (scoring formula changes, WFO logic changes, data re-exports). Always dry-run first (no `--apply`) to preview what will be deleted. Use `--reset-params` to also widen all range-based weight params back to defaults — useful when prior locks came from a DB that no longer reflects the current selection logic. Structural locks (`{"values": [...]}`) are always preserved.

*   `rescore_db.py`: Recomputes `composite_score` for all DB rows using the current scoring formula in `config.py`. Use after changing the composite metric without needing to re-run optimization from scratch.