# 8H BTC Parity Investigation — Handoff Prompt

## Context

The 8H BTC MLP strategy has a severe Python vs TradingView P&L discrepancy:
- **Python**: 54 trades, 1,400% P&L, Calmar 0.340
- **TV**: 55 trades, 2.59% P&L

The winner CSV at `results/winners/optimization_winner_strategy_mlp_scores_COINBASE_BTCUSD_8H.csv`
holds the current best params: `entry=115, exit=250, exit_conf=-80, trail=20%, exit_window=3, entry_window=3`.

Weights file: `strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_8H_bb50_bull_seed202.json`

## What We Know (Confirmed)

### Score Parity: PERFECT ✓
`check_mlp_parity.py` confirmed max|Δ|=0.000000 — Python and Pine compute identical scores.

### Trade #1 (TV#1)
- **Entry**: Dec 7T00:00Z signal → fills Dec 7T08:00Z @ $14,487.80
  - SMA3 crossunder: ~122.28 → 8.53 (below entry threshold 115) ✓
- **Exit**: Dec 15T08:00Z signal (2-bar raw crossunder in dead-zone mode, confirmation score=-246.7 < -80)
  - Fills Dec 15T16:00Z @ $17,568 (+20.06%) ✓ matches TV

**Python discrepancy for TV#1**: Python fires trailing stop at **Dec 9T08:00Z** instead of
exiting via score signal at Dec 15. Root cause: CSV has `high=$19,697` at Dec 7T16:00Z, setting
`trail_high=$19,697`, `stop=$15,757.60`. Python's `close[Dec 9T08]=15,517.02 ≤ $15,757.60` → fires.

But **TV does NOT fire** the trailing stop here, even though the exported chart data shows the same high.
TV exits cleanly via the score signal at Dec 15 (+20%).

Important correction from the Pine review: the strategy is not using a built-in stop order. It is using
manual, end-of-bar logic:

```pine
if close <= trail_high_price * (1.0 - i_trailing_stop_threshold / 100.0)
    strategy.close("long")
```

So the current Pine semantics are **close-based**, not low-based. If the exported OHLC and inferred
`trail_high_price` match what the strategy backtester actually saw, TV should submit a close order when
`close <= stop`. The fact that it did not means at least one inferred assumption still needs verification:
the backtester OHLC, the live `trail_high_price`, the live stop price, the position state/timing, or the
exact script/params pasted into TradingView.

### Trade #2 (TV#2)
- **Entry**: Dec 18T00:00Z signal → fills Dec 18T08:00Z @ $18,853.86
  - SMA3 crossunder: 334.52 → 45.25 (below entry threshold 115) ✓
  - `_dz_in_pos = TRUE` (SMA3[signal bar]=45.25 < 250) → uses raw score crossunder for exit
- **Expected exit under the current Pine code**: trailing stop should trigger on Dec 22T00:00Z
  - `trail_high = $19,175` (set at Dec 18T16:00Z bar, high=$19,175)
  - `stop = $19,175 × 0.80 = $15,340`
  - Dec 22T00:00Z: `close=$13,660.91 ≤ $15,340` → condition TRUE
  - Because this is `strategy.close()`, the expected fill is the next bar's market fill, not an intrabar
    stop-price fill.
  - But **TV holds until Sep 3, 2018 @ -61.76%**!

Under a **low-based** stop design, the same trade would have triggered earlier at Dec 20T00:00Z because
`low=$14,001 < stop=$15,340`. That is not what the current Pine code implements.

The score never recovers above exit threshold 250 after the Dec 18 entry, so the score-based
exit never fires during the 2018 bear market. The trailing stop is the only expected exit
mechanism. The open question is why the inferred close-based stop did not close the TV trade.

### Dead-zone Guard (Both TV and Python)
The dead-zone guard is understood and correct:
- Detects at entry bar: `if SMA3[signal_bar] < exit_threshold` → `use_raw = True`
- For TV#1 and TV#2: `use_raw = True` (both enter when SMA3 is deeply below 250)
- This switches exit to raw 2-bar crossunder: `score[t-2] >= 250 AND score[t-1] < 250 AND score[t] < -80`
- Python and Pine agree on this logic ✓

## Corrected Root-Cause Status

The root cause is **not confirmed yet**. The confirmed implementation fact is:

- `strategies/strategy_mlp_scores.pine` runs with `calc_on_every_tick=false`.
- The trailing high is maintained while a position is open:
  `trail_high_price := (strategy.position_size[1] <= 0) ? strategy.position_avg_price : math.max(trail_high_price, high)`.
- The trailing stop trigger is manual and close-based:
  `close <= trail_high_price * (1 - trail_pct)`.
- Python currently mirrors that same close-based trigger in `_apply_trailing_stop_with_dz()`:
  `trail_hit = close_arr[t] <= stop_price`.

Therefore, switching Pine to `strategy.exit(..., stop=...)` and Python to `low_arr[t] <= stop_price`
would be a **semantic change**, not a direct parity fix for the current implementation.

The current mismatch should be investigated as: "Why did the expected close-based stop not close the TV
trade when exported data implies `close <= stop`?" Plausible explanations:

1. **Backtester/export data discrepancy**: the OHLC seen by the strategy backtester differs from the
   plotchar/exported chart data.
2. **Stop-state mismatch**: `trail_high_price` or `stop_price` in Pine is not what the Python inference
   assumes.
3. **Script/params mismatch**: TradingView may have been running a stale pasted script or different preset.
4. **Order timing/strategy behavior**: less likely, but only worth pursuing if diagnostics prove
   `close <= stop` was true in Pine while the position stayed open.

## Decision Point: Parity Fix vs Stop-Semantics Change

There are two valid next paths. They should not be mixed.

### Path A — Preserve Current Pine Semantics and Debug Parity

Keep the current close-based manual stop and add only the minimum diagnostics needed to TradingView export:

```pine
float trail_stop_price = trail_high_price * (1.0 - i_trailing_stop_threshold / 100.0)
```

Export enough stop state to verify the Dec 18 trade, subject to the 64-plot export limit:
- `trail_high_price / close`
- `trail_stop_price / close`

Do not spend plot slots on close or position state unless later evidence requires it. Close is already in
the OHLC export, and position size should not vary unexpectedly for this single long-only strategy path.
The close-based hit condition can be derived offline as `close <= trail_stop_price`.

Expected under the current Pine code:
- TV#2 should have exported `trail_stop_price >= close` on Dec 22T00:00Z if the inferred stop state is correct.
- If `trail_stop_price < close`, the issue is data/state inference, not `strategy.close()`.
- If `trail_stop_price >= close` and TV still holds the trade, then investigate Pine order behavior or
  script freshness.

### Path B — Intentionally Change to Low-Based Intrabar Stops

If the desired trading behavior is "stop out when the bar's low breaches the stop", then replace the
manual stop with `strategy.exit()` and update Python to use `low_arr[t] <= stop_price`.

This is a strategy-design change. It will likely improve catastrophic loss handling, but it changes trigger
timing, fill price, and optimized parameter meaning. Thresholds must be re-swept after the change.

## Low-Based Stop Change (Only If Path B Is Chosen)

**Replace the manual trailing stop with `strategy.exit()`** — Pine's standard built-in stop mechanism.
This uses intrabar evaluation (`low <= stop_price`) instead of bar-close evaluation (`close <= stop_price`).

### Pine change (in `strategies/strategy_mlp_scores.pine` around lines 993-995):

**Current code**:
```pine
if (i_trailing_stop_threshold > 0 and strategy.position_size > 0 and timeCondition)
    if close <= trail_high_price * (1.0 - i_trailing_stop_threshold / 100.0)
        strategy.close("long")
```

**Replace with**:
```pine
if (i_trailing_stop_threshold > 0 and strategy.position_size > 0 and not na(trail_high_price) and timeCondition)
    strategy.exit("trail_stop", from_entry="long", stop=trail_high_price * (1.0 - i_trailing_stop_threshold / 100.0))
```

This intentionally changes the trigger from `close <= stop` (bar close) to `low <= stop` (intrabar).
With `fill_orders_on_standard_ohlc=true`, `strategy.exit()` fires when the bar's LOW
crosses the stop price, and fills AT the stop price (not the next bar's open).

### Python change (in `strategies/strategy_mlp_scores.py`, `_apply_trailing_stop_with_dz()`):

**Current** (line 277):
```python
trail_hit = close_arr[t] <= stop_price
```

**Replace with**:
```python
trail_hit = low_arr[t] <= stop_price
```

The function signature needs `low_arr` added (replacing or alongside `close_arr`):

Current signature:
```python
def _apply_trailing_stop_with_dz(entry_raw_arr, exit_raw_arr, exit_raw_fallback_arr,
                                   score_exit_arr, exit_threshold,
                                   close_arr, high_arr, trail_stop_pct):
```

Updated: add `low_arr` parameter, change `trail_hit` to use `low_arr[t]`.

Also need to update `_apply_trailing_stop` (the non-dead-zone version, imported from
`strategies/library_activation_scores.py`) to use `low_arr` instead of `close_arr`.

In `_score_to_signals()` around line 500:
- Add: `low_arr = df['low'].values.astype(np.float64)`
- Pass `low_arr` instead of `close_arr` to both trailing stop function calls

### Impact If Path B Is Chosen

- **TV#2**: With `low <= stop` trigger, the stop would fire at Dec 20T00:00Z
  (`low=$14,001 < stop=$15,340`) and TV would fill at the stop price. P&L ≈ -18% (vs current -61.8%)
- **TV#1**: The $19,697 spike is in Python's data. With `low <= stop` trigger:
  - Python: Dec 8T08:00Z `low=$13,788.99 < stop=$15,757.60` → fires 1 bar earlier
  - TV (if corrected data has lower high): stop lower, may not fire before score-based exit
  - This is an acceptable discrepancy caused by data quality difference between CSV and TV backtester
- **Net IS effect**: Catastrophic -61.8% losses become -18% losses → IS Calmar should improve significantly
- **Re-optimization required**: The `close <= stop` parameters were tuned to close-based semantics.
  With `low <= stop`, the optimal trail threshold will likely be different. Must re-sweep.

### Fill Price Consideration

Pine with `strategy.exit()` fills at the stop price (intrabar). Python currently fills at
`open[t+1]` for all exits (via the position array / pct_change mechanism in `calculate_metrics`).

For the Dec 20T00:00Z example:
- stop_price = $15,340
- open[Dec 20T08:00Z] = $16,551 > $15,340

Python would compute exit return as: close[Dec 19T16] ($17,838) → close[Dec 20T00] ($16,551) ≈ -7.2%
TV would compute exit return as: fill at $15,340 from entry $18,853 ≈ -18.6%

This fill price discrepancy means Python will overestimate P&L vs TV for trailing stop exits.
To model Path B properly, `calculate_metrics()` would need to handle trailing stop fill prices separately.

## The 64-Plot Limit

The Pine strategy is at exactly 64 plotchars (TV's hard limit). To add diagnostics for
`trail_high_price` and `trail_stop_price`, one or two safe-to-remove plotchars must be freed first.
If only one slot is available, prioritize `trail_stop_price / close`; it directly answers whether
the close-based stop condition should have fired.

**Safe to remove** (not MLP inputs or condition gates):
- `gc_hband` — `gc_position` remains exported as the actual MLP feature
- `gc_lband` — `gc_position` remains exported as the actual MLP feature

Do not remove `macd_bullish_norm`, `rsid_norm`, or `stoch_bot_norm`. They have
nonzero MLP weights even though an older perceptron workflow locked their scalar
weight parameters to zero; omitting them makes Python substitute zero and breaks
score parity.

If debugging Path A before changing semantics, free one or two plot slots and add:
```pine
plotchar(trail_high_price / close, "th", "•", location.bottom, color.new(color.white, 100), size=size.tiny)
plotchar(trail_stop_price / close, "ts", "•", location.bottom, color.new(color.white, 100), size=size.tiny)
```
These export ratios should be present while in position and `na` when flat.

## What Needs To Happen Next

1. **Choose the next path**:
   - Path A: preserve current close-based Pine semantics and add diagnostics.
   - Path B: intentionally change strategy semantics to low-based `strategy.exit()` stops.

2. **If Path A, add temporary Pine diagnostics** for `trail_high_price / close` and
   `trail_stop_price / close` if the plot budget allows. If only one slot can be freed,
   prioritize `trail_stop_price / close`.

3. **Paste diagnostic Pine into TradingView**, re-export data:
   - "Export chart data" → `data/mlp/COINBASE_BTCUSD, 480.csv`
   - "Export strategy trades" → `~/Downloads/MLPScores_COINBASE_BTCUSD_<date>.csv`

4. **Re-run compare_tv_trades.py** to check the diagnosed mismatch:
   ```bash
   python3 tools/compare_tv_trades.py \
     --tv-trades ~/Downloads/MLPScores_COINBASE_BTCUSD_<date>.csv \
     --data "data/mlp/COINBASE_BTCUSD, 480.csv"
   ```

5. **If diagnostics prove Pine close-stop state is correct but TV still does not close**, investigate
   `strategy.close()` behavior/order timing or stale script state.

6. **If Path B is chosen**, implement the Pine/Python low-based stop change, update tests, then re-sweep
   thresholds because stop semantics changed:
   ```bash
   python3 tools/run_mlp_deep_sweep.py \
     --asset COINBASE_BTCUSD --timeframes 8H \
     --samples 80000 --promote \
     --extra-weights strategies/params/mlp/mlp_weights_COINBASE_BTCUSD_8H_bb50_bull_seed202.json
   ```

7. **Run `/mlp-results`** and include the table in the commit message (per memory: always required for MLP promotions).

8. **After user confirms TV behavior**: Commit with gate condition satisfied.

## Current File State

- **Pine**: `strategies/strategy_mlp_scores.pine` — uses manual `strategy.close("long")` trailing stop with close-based end-of-bar trigger.
- **Python**: `strategies/strategy_mlp_scores.py` — uses `close_arr[t] <= stop_price`, matching the current Pine trigger.
- **Winner CSV**: `results/winners/optimization_winner_strategy_mlp_scores_COINBASE_BTCUSD_8H.csv` — currently holds params from the deep sweep run earlier (entry=115, exit=250, trail=20%). This was swept with the current `close <= stop` semantics and will need re-sweeping only if stop semantics change.
- **Data**: `data/mlp/COINBASE_BTCUSD, 480.csv` — exported Jun 22 with the corrected 55-feature Pine strategy. Has the $19,697 spike at Dec 7T16:00Z (potential data quality issue from Coinbase).

## Key Data Points for Verification

Under current close-based semantics, these are the key expectations to verify:
- TV#2 (Dec 18 entry): exported `trail_stop_price / close` should be `>= 1.0` on Dec 22T00:00Z
  if exported OHLC and inferred `trail_high_price` match the actual Pine state.
- Under a low-based Path B change, TV#2 would instead trigger earlier around Dec 20T00:00Z.
- TV#1 (Dec 7 entry): behavior depends heavily on whether Pine's actual backtester state includes the
  $19,697 Dec 7T16:00Z high spike.
- TV#25 (Nov 2021 entry): another catastrophic holddown; use the same diagnostics before assuming the
  same cause.

The final gate condition: **user must confirm Pine behavior in TradingView** before committing or promoting
new MLP results.