# Macro/Sentiment Conditions — Integration Spec

**Purpose:** Add macro and cross-asset sentiment signals as exported components in the activation-score Pine script, so the Python optimizer can learn weights for them alongside existing technical components.

---

## 1. Rationale

BTC is a risk-on asset with measurable correlations to global macro conditions:

- **USD strength (DXY):** Inverse correlation with BTC. A weakening dollar reduces the opportunity cost of holding non-yielding assets and increases global liquidity. A rising DXY signals risk-off rotation.
- **Equity volatility (VIX):** Elevated VIX = risk-off. BTC has historically sold off with equities during VIX spikes (2020 March, 2022 etc.). A declining VIX = risk-on regime.
- **US10Y yield direction:** Rising yields increase the discount rate, compressing speculative asset valuations. Rate-of-change matters more than absolute level. Yield curve slope (10Y-2Y) can signal recession risk.
- **Global M2 liquidity (M2SL / WM2NS):** BTC price leads global M2 by ~3 months (Raoul Pal thesis). Expanding M2 = tailwind. Already partially covered by existing M2/M3 components, but those use EU+CN+US money supply inside the strategy's LibraryMoneySupply. A separate USD-only M2 rate-of-change may add orthogonal signal.
- **BTC dominance (BTC.D):** Rising dominance = altcoin capital rotating into BTC, often a BTC-specific bullish signal. Falling dominance = risk appetite moving outward (can precede BTC tops or ETH/altcoin rallies).
- **Gold (XAUUSD):** Gold rising alongside BTC = macro liquidity/inflation driven rally (genuine tailwind). Gold rising while BTC falls = flight to safety (bearish divergence). Gold rate-of-change is a useful risk regime indicator.
- **SPY/equities momentum:** BTC correlates with SPY on risk-off days. SPY trend as a regime filter prevents buying into macro drawdowns.
- **Copper (HG1!):** "Dr. Copper" — a proxy for global growth expectations. Rising copper = global expansion → risk-on.

Adding 3–6 of these as normalized exported columns gives the optimizer cross-asset regime context it currently lacks.

---

## 2. Prioritized Macro/Sentiment Conditions

Priority order reflects: (a) strength of empirical BTC correlation, (b) data availability on TradingView free tier, (c) avoidance of overlap with existing components.

---

### P1 — DXY Rate of Change

| Field | Value |
|---|---|
| TV ticker | `TVC:DXY` |
| Pine fetch | `request.security("TVC:DXY", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 14-bar rate-of-change, normalised to -1/0/+1 via 3-way sign: `dxy_roc > 0 ? -1 : dxy_roc < 0 ? 1 : 0` (inverse polarity: DXY up = BTC bearish) |
| Expected weight sign | **Negative** (optimizer likely finds weight < 0 because the raw export already inverts sign; or export raw ROC percentile and let optimizer set sign) |
| Limitation | DXY is a 24h forex market; available on all TV timeframes. No gaps. Daily fetch forward-filled onto sub-daily bars. |
| CSV column | `dxy_roc_norm` |

**Recommended export:** Rolling 252-bar percentile rank (0–1) of the 14-bar ROC, then invert: `1 - percentile_rank(dxy_roc, 252)`. Produces 0 when DXY is rising fast (BTC bearish) and 1 when DXY is falling fast (BTC bullish). Percentile is more stationary than raw ROC over multi-year data.

---

### P2 — VIX Level / Regime

| Field | Value |
|---|---|
| TV ticker | `CBOE:VIX` |
| Pine fetch | `request.security("CBOE:VIX", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 252-bar percentile rank of VIX level, inverted: `1 - percentile_rank(vix, 252)`. 0 = high fear / risk-off; 1 = low fear / risk-on. |
| Expected weight sign | **Positive** (higher value = calmer market = more BTC bullish) |
| Limitation | VIX is calculated from US equity options; is 0 on weekends and US market holidays in TradingView. Must use `nz(vix_val, vix_val[1])` to forward-fill. On 4H/8H charts many bars will be NA — mitigate with `gaps=barmerge.gaps_off`. |
| CSV column | `vix_pctrank_inv` |

**Note:** Also consider exporting a binary `vix_spike` flag: `vix > 30 ? -1 : 0`. This penalises entries during acute risk-off events regardless of the percentile rank.

---

### P3 — BTC Dominance Direction

| Field | Value |
|---|---|
| TV ticker | `CRYPTOCAP:BTC.D` |
| Pine fetch | `request.security("CRYPTOCAP:BTC.D", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 3-way sign of 7-bar ROC: `btcd_roc > 0 ? 1 : btcd_roc < 0 ? -1 : 0` |
| Expected weight sign | **Positive** (rising BTC.D = capital rotating into BTC = bullish for BTC longs) |
| Limitation | Available 24/7, no gaps. Data starts ~2013. Reliable on all timeframes. |
| CSV column | `btc_dom_roc_sign` |

---

### P4 — US10Y Rate of Change (direction)

| Field | Value |
|---|---|
| TV ticker | `TVC:US10Y` |
| Pine fetch | `request.security("TVC:US10Y", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 3-way sign of 14-bar ROC: `us10y_roc > 0 ? -1 : us10y_roc < 0 ? 1 : 0` (rising yields = bearish BTC, so export inverts) |
| Expected weight sign | **Positive** after inversion (inverted = higher value when yields falling = bullish for BTC) |
| Limitation | US bond market; no weekend data. Forward-fill with `gaps=barmerge.gaps_off`. Data available from 1961 on TV. Absolute yield level is less informative than direction; use ROC. |
| CSV column | `us10y_roc_inv_sign` |

---

### P5 — SPY Trend (Risk-On Regime Filter)

| Field | Value |
|---|---|
| TV ticker | `AMEX:SPY` or `SP:SPX` |
| Pine fetch | `request.security("AMEX:SPY", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | Whether SPY is above its 200-day EMA: `spy_close > ta.ema(spy_close, 200) ? 1 : -1` |
| Expected weight sign | **Positive** (SPY above 200 EMA = risk-on = BTC bullish regime) |
| Limitation | US market hours only; weekend/holiday gaps. Forward-fill. Data from 1993. |
| CSV column | `spy_above_200ema` |

---

### P6 — Gold Rate of Change (Liquidity Proxy)

| Field | Value |
|---|---|
| TV ticker | `TVC:GOLD` or `OANDA:XAUUSD` |
| Pine fetch | `request.security("TVC:GOLD", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 252-bar percentile rank of 21-bar ROC. High value = gold rising quickly = liquidity/inflation tailwind. |
| Expected weight sign | **Positive** (gold rally often co-occurs with BTC rally during liquidity expansions; ambiguous during flight-to-safety, hence percentile rank preferred over raw) |
| Limitation | 24h forex-adjacent market; minimal gaps. Ambiguous polarity (can be flight-to-safety OR inflation hedge) — optimizer should determine sign. |
| CSV column | `gold_roc_pctrank` |

---

### P7 — BTC/SPX Rolling Correlation (Risk Regime Index)

| Field | Value |
|---|---|
| TV ticker | `SP:SPX` |
| Pine fetch | `request.security("SP:SPX", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 30-bar rolling correlation of BTC 1-bar ROC vs SPX 1-bar ROC: `ta.correlation(ta.roc(close, 1), ta.roc(spx_close, 1), 30)`. Range is naturally -1 to +1. |
| Expected weight sign | **Uncertain / optimizer decides.** High positive correlation = BTC acting risk-on (correlated to equities). Low/negative = decoupled or risk-off safe haven. The optimizer may find that *either* high or low correlation is useful depending on the regime. |
| Limitation | SPX has no weekend data; `gaps_off` forward-fills. Correlation stabilizes after ~30 bars, so early history may be noisy. Requires at least 2 bars of both series to compute. |
| CSV column | `btc_spx_corr_30` |

**Why this is useful beyond the other macro signals:**

DXY, VIX, US10Y, and SPY each measure *external conditions* — they tell you the state of macro markets. This signal is different: it directly measures whether BTC is *responding* to macro conditions on a given date. There are extended periods (2020 DeFi summer, 2023 post-FTX recovery) where BTC rallied independently of SPX despite a high-VIX, high-DXY environment. A strategy using only macro conditions would have missed those entries. The rolling correlation catches this: when correlation drops toward 0, BTC is in idiosyncratic mode and macro signals should be down-weighted. When it's high (+0.7), BTC is trading like a risk asset and macro signals matter more.

**Pine implementation (3 lines):**
```pine
spx_close  = request.security("SP:SPX", "D", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)
btc_spx_corr_30 = ta.correlation(ta.roc(close, 1), ta.roc(nz(spx_close, spx_close[1]), 1), 30)
plotchar(btc_spx_corr_30, "btc_spx_corr_30", color=color.gray, display=display.data_window)
```

The correlation is already in [-1, +1] so no further normalization is needed — it goes directly into the activation score.

**Note on combining with other signals:**

If the optimizer assigns a meaningful weight to `btc_spx_corr_30`, it does *not* replace DXY/VIX/SPY. It complements them. A future enhancement would be to use the correlation as a *regime gate* (only apply macro weights when correlation is above a threshold), but that's not compatible with the current linear weighted-sum architecture. For now, let the optimizer treat it as a linear feature.

---

### P8 — Global M2 (USD-only, monthly FRED)

| Field | Value |
|---|---|
| TV ticker | `FRED:M2SL` (weekly) or `FRED:WM2NS` (weekly seasonally adjusted) |
| Pine fetch | `request.security("FRED:WM2NS", "W", close, gaps=barmerge.gaps_off, lookahead=barmerge.lookahead_off)` |
| Export value | 52-bar (1Y) percentile rank of 13-bar ROC of M2. 1 = M2 growing fast, 0 = contracting. |
| Expected weight sign | **Positive** (expanding money supply = BTC bullish) |
| Limitation | Weekly resolution. On daily and sub-daily charts, this value changes only once per week — many repeated values in the export. The optimizer can still learn a weight; it just has lower frequency variation. Already partially overlaps with existing M2/M3 components from LibraryMoneySupply — include only if correlation analysis shows orthogonal signal. |
| CSV column | `m2_wm2ns_roc_pctrank` |

---

## 3. Implementation Complexity Per Signal

The general pattern for every signal is:
1. `request.security(...)` to fetch the external series
2. Optional: compute ROC, percentile rank, or correlation
3. Export via `plotchar(value, "column_name", display=display.data_window)`
4. Add `i_w_*` input weight (default 0.0)
5. The optimizer finds the weight

| Signal | Pine logic needed | Approx. lines |
|---|---|---|
| BTC/SPX correlation | `ta.roc()` × 2 + `ta.correlation()` | 3 |
| BTC.D ROC sign | `ta.roc()` + 3-way sign | 3 |
| SPY above 200 EMA | `ta.ema()` + comparison | 2 |
| DXY ROC sign | `ta.roc()` + 3-way sign + invert | 3 |
| US10Y ROC sign | `ta.roc()` + 3-way sign + invert | 3 |
| VIX percentile | `ta.roc()` optional + percentile loop | ~8 |
| Gold ROC percentile | `ta.roc()` + percentile loop | ~8 |
| M2 WM2NS ROC pctrank | `ta.roc()` + percentile loop | ~8 |

The percentile loop is reusable — define `f_pctrank()` once and call it for VIX, Gold, and M2. Total new Pine code across all 8 signals: approximately 40-50 lines.

---

## 5. Normalisation Plan

All normalised values should land in the range **-1 to +1** or **0 to +1** consistently. The existing components use 3-way sign (-1/0/+1) for discrete events (like `m2_div_osc`) and continuous floats for continuous signals. Follow the same pattern:

### Method A — 3-way sign (for directional signals)
```pine
// Use when the direction is the signal, not the magnitude
sign_val = roc > 0 ? 1 : roc < 0 ? -1 : 0
```
Apply to: DXY ROC, US10Y ROC, BTC.D ROC, SPY above/below 200 EMA.

### Method B — Rolling percentile rank (0 to 1, for magnitude signals)
```pine
// Approximate percentile rank over N bars using a loop
f_pctrank(series float src, simple int len) =>
    float count = 0.0
    for i = 0 to len - 1
        if src[i] <= src
            count += 1.0
    count / len
```
Apply to: VIX (inverted), Gold ROC, M2 ROC. Produces stationary output across all market regimes.

### Method C — Z-score clamped to [-1, +1]
```pine
// Use when mean-reversion around a level is the signal
z = (src - ta.sma(src, len)) / ta.stdev(src, len)
z_clamped = math.max(-1.0, math.min(1.0, z / 2.0))  // divide by 2 to put ±2σ at ±1
```
Apply to: VIX if percentile feels too slow to react. Not recommended as primary method — percentile rank is more robust to regime shifts.

### Forward-fill convention
For any daily/weekly source on sub-daily chart bars, always use `gaps=barmerge.gaps_off` in `request.security()`. This causes Pine to carry the last known value forward automatically. Do **not** use `lookahead=barmerge.lookahead_on` — this would introduce future leak.

---

## 6. Integration Plan

### 4.1 Pine plotchar column names

Follow the existing convention from `strategy_activation_scores.pine`:
- **Non-debug columns** (main export for Python): lowercase with underscores, no prefix. These map directly to `i_w_*` params.
- **Debug columns** (debug indicator only): prefixed with `DB_`, title-case.

| Component | plotchar title (main export) | plotchar title (debug) |
|---|---|---|
| BTC/SPX 30-bar correlation | `btc_spx_corr_30` | `DB_BTC_SPX_Corr` |
| DXY ROC norm | `dxy_roc_norm` | `DB_DXY_ROC` |
| VIX percentile inv | `vix_pctrank_inv` | `DB_VIX_Pct` |
| BTC dominance ROC sign | `btc_dom_roc_sign` | `DB_BTCD_ROC` |
| US10Y ROC inv sign | `us10y_roc_inv_sign` | `DB_US10Y_ROC` |
| SPY above 200 EMA | `spy_above_200ema` | `DB_SPY_200` |
| Gold ROC percentile | `gold_roc_pctrank` | `DB_Gold_ROC` |
| M2 WM2NS ROC pctrank | `m2_wm2ns_roc_pctrank` | `DB_M2_WM2NS` |

Add to `strategy_activation_scores.pine` plotchar block (lines ~403–427):
```pine
plotchar(btc_spx_corr_30,       title="btc_spx_corr_30",        color=color.gray, display=display.data_window)
plotchar(dxy_roc_norm,          title="dxy_roc_norm",          color=color.gray, display=display.data_window)
plotchar(vix_pctrank_inv,       title="vix_pctrank_inv",        color=color.gray, display=display.data_window)
plotchar(btc_dom_roc_sign,      title="btc_dom_roc_sign",       color=color.gray, display=display.data_window)
plotchar(us10y_roc_inv_sign,    title="us10y_roc_inv_sign",     color=color.gray, display=display.data_window)
plotchar(spy_above_200ema,      title="spy_above_200ema",       color=color.gray, display=display.data_window)
plotchar(gold_roc_pctrank,      title="gold_roc_pctrank",       color=color.gray, display=display.data_window)
// Optional — only add if M2 correlation analysis shows orthogonal signal:
// plotchar(m2_wm2ns_roc_pctrank, title="m2_wm2ns_roc_pctrank", color=color.gray, display=display.data_window)
```

### 4.2 Pine score formula additions

In `LibraryLongEntry.pine` (or wherever `calculate_activation_score_poc` is defined), add the new terms to the weighted sum. Each term follows the existing pattern:

```pine
// Inside calculate_activation_score_poc, normalise then weight:
dxy_norm    = dxy_roc_norm      // already -1/0/+1
vix_norm    = vix_pctrank_inv   // already 0-1
btcd_norm   = btc_dom_roc_sign  // already -1/0/+1
us10y_norm  = us10y_roc_inv_sign
spy_norm    = spy_above_200ema  // -1 or +1
gold_norm   = gold_roc_pctrank  // 0-1

score += i_w_dxy       * dxy_norm
score += i_w_vix       * vix_norm
score += i_w_btc_dom   * btcd_norm
score += i_w_us10y     * us10y_norm
score += i_w_spy       * spy_norm
score += i_w_gold      * gold_roc
```

### 4.3 New `i_w_*` input params in Pine

Add to the `group_neural_activation_weights` section:
```pine
i_w_btc_spx_corr = input.float(0.0, step=0.1, title="BTC/SPX Correlation Weight", group=group_neural_activation_weights, display=display.none)
i_w_dxy     = input.float(0.0, step=0.1, title="DXY ROC Weight",         group=group_neural_activation_weights, display=display.none)
i_w_vix     = input.float(0.0, step=0.1, title="VIX Percentile Weight",   group=group_neural_activation_weights, display=display.none)
i_w_btc_dom = input.float(0.0, step=0.1, title="BTC Dominance ROC Weight",group=group_neural_activation_weights, display=display.none)
i_w_us10y   = input.float(0.0, step=0.1, title="US10Y ROC Weight",        group=group_neural_activation_weights, display=display.none)
i_w_spy     = input.float(0.0, step=0.1, title="SPY 200EMA Weight",       group=group_neural_activation_weights, display=display.none)
i_w_gold    = input.float(0.0, step=0.1, title="Gold ROC Weight",         group=group_neural_activation_weights, display=display.none)
```
Default to 0.0 so existing behaviour is unchanged until the optimizer assigns values.

### 4.4 New entries in `params_strategy_activation_scores_1D.json`

Add these ranges after the existing weight entries. Initial ranges are intentionally wide; narrow after first optimization pass:
```json
"i_w_btc_spx_corr": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_dxy": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_vix": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_btc_dom": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_us10y": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_spy": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
},
"i_w_gold": {
    "start": -20.0,
    "stop": 20.0,
    "step": 2.0
}
```

### 4.5 Python `library_activation_scores.py` additions

In `_prepare_features` (called by both CPU and GPU paths), add columns in the **same order** as the `i_w_*` keys appear in the params JSON. Append new feature columns at the end:

```python
# In _prepare_features, add after existing feature columns:
"btc_spx_corr_30",
"dxy_roc_norm",
"vix_pctrank_inv",
"btc_dom_roc_sign",
"us10y_roc_inv_sign",
"spy_above_200ema",
"gold_roc_pctrank",
```

The normalisation in Pine handles the 0–1 / -1–+1 scaling, so Python reads the pre-normalised values directly from CSV with no additional transform (same pattern as existing components like `m2_div_osc` which is already a 3-way sign).

### 4.6 Handling different update frequencies

| Frequency | Behaviour on 1D bars | Behaviour on sub-daily bars |
|---|---|---|
| VIX (US market days only) | 1 value/day, no weekends | `gaps=barmerge.gaps_off` fills forward; ~70% of 4H bars will carry previous value |
| DXY/Gold (forex, ~24h) | Daily values; minor gaps on holidays | Minimal gaps; `gaps_off` handles |
| SPY (US market hours) | 1 value/day | Same as VIX; forward-filled on off-hours bars |
| US10Y (bond market) | Daily | Forward-filled |
| BTC.D (crypto, 24/7) | Full coverage | Full coverage; no gaps |
| M2/WM2NS (weekly FRED) | 1 value/week; 6 repeated daily bars | 1 value/week; ~168 repeated hourly bars |

For Python backtesting: since CSV is exported at the chart timeframe (1D), all values appear once per bar. The forward-fill is done by Pine before export, so Python sees no gaps — all rows have a valid float value. No Python-side forward-fill needed.

**Important:** When re-exporting the CSV after adding these columns, verify that new columns contain no `NaN` rows in the Python data (use `df[new_cols].isna().sum()` to check). If NaNs are present in early history (e.g. BTC.D data pre-2015), either filter the backtest start date or use `nz(val, 0)` in Pine before plotchar.

---

## 7. Implementation Order

1. **Add `request.security()` calls** in Pine for each source. Group them near the top of the script after existing imports.
2. **Compute normalised values** using the methods specified in Section 3.
3. **Add `plotchar` lines** (Section 4.1). Export CSV from TV and confirm new columns appear.
4. **Add to score formula** in the activation score library (Section 4.2).
5. **Add `i_w_*` inputs** to Pine (Section 4.3).
6. **Update params JSON** (Section 4.4).
7. **Update `_prepare_features`** column list in Python (Section 4.5) — maintain key order parity with JSON.
8. **Run correlation check:** Before running full optimization, print `df[new_cols].corr()` against `df['activation_score_poc']` to confirm signal is not purely redundant.
9. **Wide-range optimization pass** with new weights included; narrow ranges after first pass.

---

## 8. Scope Exclusions (not recommended at this stage)

- **Fear & Greed Index** (alternative.me): not available in TradingView natively; requires custom Pine HTTP fetch — not supported in strategy scripts.
- **Funding rates / open interest:** Not available on TradingView free tier for direct `request.security()` use.
- **Yield curve (10Y-2Y):** `TVC:US10Y - TVC:US02Y` is computable in Pine but adds complexity for limited marginal gain over US10Y direction alone. Revisit if US10Y weight proves significant.
- **Copper (HG1!):** Overlaps strongly with SPY as a growth proxy. Deferred to avoid redundant features.