# Investigation: M2 Indicator 1D Performance Regression **Last Updated:** 2025-09-09 ## 1. The Problem After refactoring the M2 Leading Indicator logic in `MainStrategy.pine` to support multiple timeframes (`1D` and `12H`), a severe performance regression was observed on the `1D` timeframe. The goal of the refactor was to improve `12H` performance without altering the `1D` baseline. The persistence of the `1D` regression indicates that despite several fixes, the current logic does not perfectly replicate the original, performant baseline logic. The core of the issue lies in the complex chain of calculations involving the M2 indicator: 1. Fetching raw M2 data. 2. Applying historical offsets (`tiny`, `short`, `medium`, `long`). 3. Applying smoothing (or not). 4. Scaling the values to a 0-100 range. 5. Calculating differences and slopes between current and future values (`N_bars_out`). 6. Using these final values in entry/exit conditions. A subtle change in any of these steps has proven to have a dramatic impact on the final strategy performance. --- ## 2. Attempts and Results This section documents the iterative attempts to fix the regression. * **Attempt 1: Introduce Timeframe-Specific Inputs** * **Action:** Replaced single `input.int` for M2 `shortOffset` and `smoothingLength` with `_1D` and `_12H` versions, selected via `utils.f_getParamForTimeframe_int`. * **Result:** **Compile Error.** The `utils.f_ma` function (using `ta.ema`) requires a `simple int` for its length, but our new variable was a `series int`. * **Attempt 2: Use Dynamic Smoothing Function** * **Action:** Replaced `utils.f_ma` with `utils.f_apply_dynamic_smoothing` to handle the `series int` length. * **Result:** **1D Regression & 12H Runtime Error.** The custom `_dynamic_ema` implementation, while functional, behaved differently from the optimized built-in `ta.ema`, causing the 1D regression. A logic error also caused a negative lookback (`-18 bars`) on the 12H chart. * **Attempt 3: Fix Runtime Error & Restore Logic** * **Action:** Used `math.max(0, ...)` to prevent the negative lookback. Restored a `timeframe_divisor` for `tinyOffset` that was accidentally removed. * **Result:** **12H Fixed, 1D Unchanged.** The 12H runtime error was resolved, but the 1D performance regression remained. * **Attempt 4: Conditional Smoothing Logic** * **Action:** Implemented an `if timeframe.period == "D"` block to use the original `utils.f_ma` on 1D and the dynamic version on other timeframes. * **Result:** **Compile Error.** The compiler still identified the length parameter as a `series int` even within the `if` block. * **Attempt 5: Use Explicit Input Variables** * **Action:** Modified the `if/else` block to use the `simple int` `_1D` and `_12H` input variables directly. * **Result:** **Regression Worsened.** This indicated a logic error in how the variables were being used or in other dependent calculations. * **Attempt 6: Correct Variable Usage in `ta.rising`** * **Action:** Fixed a copy-paste bug where the rising/falling status of `m2_smoothed_tinyOffset` was being calculated using `m2_smoothed_shortOffset`. * **Result:** **Slight Improvement.** Performance improved but was still an order of magnitude worse than the baseline. * **Attempt 7: Correct Smoothing Application** * **Action:** Corrected logic where `micro`, `mini`, and `tiny` offsets were being incorrectly smoothed using the `shortOffset`'s smoothing length. Restored them to their original, unsmoothed state. * **Result:** **Slight Improvement.** Another small gain, but still far from the baseline. * **Attempt 8: Correct Order of Operations (Scaling)** * **Action:** Identified that the `f_scale_ToRange` function was being called on the *current* M2 value instead of the *historically-offsetted* M2 value for the `tinyOffset`. Corrected the order of operations. * **Result:** **No Improvement.** The regression remains, indicating this was not the root cause. * **Attempt 9: Forensic Debugging with Side-by-Side Plots** * **Action:** Created `IndicatorM2LI_Debug.pine` to plot the calculation pipeline for both the baseline and current logic. * **Result:** **SUCCESS.** Visual analysis immediately revealed the first point of divergence: The `Current` logic was failing to apply the initial smoothing to the `tiny`, `mini`, and `micro` offsets, whereas the `Baseline` logic did. This was the root cause of the entire performance regression. The fix is to re-apply this smoothing in `MainStrategy.pine`. * **Attempt 10: Isolated Fixes & Re-evaluation** * **Action (Fix A - Smoothing):** Applied only the initial smoothing fix to the `tiny`, `mini`, and `micro` offsets. * **Result:** Performance improved from ~8k% to ~12,000%. A positive but incomplete step. * **Action (Fix B - Historical Lookup):** Applied only the final difference calculation fix (the historical lookup). * **Result:** Performance improved to **12,373%**. * **Action (Fix A + B):** Applied both fixes together. * **Result:** Performance was ~12,700%. * **Conclusion:** While both fixes show individual improvements, none come close to restoring the baseline of 381k%. The combined effect is not significantly better than individual fixes. This proves that our investigation has been too narrowly focused and we are missing a more fundamental bug in the M2 calculation pipeline or in how its outputs are used. * **Attempt 11: Unified Fix based on Debugger Analysis** * **Action:** Based on a detailed analysis of the debug indicator, a unified fix was created to replicate the baseline's sequential logic (Smooth -> Offset & Scale -> Calculate Difference). * **Result:** Performance was **13,657%**. This is the current best result so far, but still significantly below the 381k% baseline. * **Key Insight from Debugger:** A detailed analysis of the debugger output revealed that while some variables (like `...Slope`) now match the baseline, others (`FINAL...Diff` and `is...Rising/Falling`) do not. This points to very specific, subtle bugs in the original baseline code that were "fixed" during our refactoring, inadvertently changing the strategy's behavior. --- ## 3. Learnings 1. **Fragility of Calculation Chains:** The M2 indicator's logic is a multi-stage pipeline. A minor deviation in an early stage (e.g., `_dynamic_ema` vs. `ta.ema`) causes significant divergence in the final output. 2. **Compiler Strictness:** The `series` vs. `simple` type system is a critical hurdle. The compiler does not infer context (e.g., inside an `if timeframe.period == "D"` block), requiring explicit variable usage. 3. **Order of Operations is Paramount:** The sequence of smoothing, offsetting, and scaling is not interchangeable. The original implementation's precise order, including variable re-assignments, must be replicated exactly. 4. **Latent Bug Dependency:** The performance of the original strategy may be dependent on subtle bugs or logical inconsistencies in its calculations. "Fixing" these bugs during a refactor can unintentionally alter the behavior and degrade performance, making a perfect 1:1 replication of the buggy logic a necessary step for verification. --- ## 5. Failed Avenue: The "Data Granularity Mismatch" Hypothesis A major hypothesis was that the root cause of the regression was a fundamental mismatch between the data's granularity and the chart's timeframe. * **Observation:** The M2/M3 money supply data is sourced at a daily (`"D"`) resolution. * **Hypothesis:** On timeframes shorter than 1D (e.g., 12H), this daily data creates a "stair-step" pattern (the same value is repeated for multiple bars). Applying standard smoothing algorithms to this non-continuous data may be creating unintended artifacts and is likely the source of the performance degradation. The original strategy worked because the data granularity (1D) matched the chart timeframe (1D). * **Proposed Solution:** The proposed solution was to create a "pre-smoothed" M2 data source by applying a light EMA to the raw 1D data first, providing a clean, continuous signal to all timeframes. ### Conclusion: Hypothesis Disproven This avenue was pursued extensively. The `f_get_presmoothed_m2` function was created in `LibraryMoneySupply`, and the `MainStrategy` was modified to use this new data source. **Result:** This approach was a definitive failure. Using the pre-smoothed data resulted in a catastrophic performance regression on **all timeframes**, including 1D. **Learning:** The strategy's high performance is inextricably linked to the raw, "stair-stepped" nature of the daily M2 data. The logic is tuned to these specific artifacts. **Any attempt to pre-smooth or otherwise "clean" the M2 data source should be considered a known dead end.** --- ## 6. Final Breakthrough and Resolution After numerous failed attempts to replicate the baseline performance, a final forensic analysis was conducted. 1. **Discovery of the True Baseline:** It was discovered that `PreviousMainStrategy.pine` was not the correct baseline. A version from TradingView (`v36`) was identified as the true source of the 318k% performance. 2. **Identification of Critical Differences:** A file comparison between `v36` and the non-performant `PreviousMainStrategy.pine` revealed the root cause. The 318k% performance was dependent on a specific, non-intuitive order of operations in the M2 calculation pipeline: * **Correct (318k%) Pipeline:** **Smooth -> Scale -> Offset**. The `v36` code first smoothed the M2 data, then scaled the *entire series* using `f_scale_ToRange`, and only then did subsequent calculations perform historical lookups (`[...]`) on the already-scaled data. * **Incorrect (Regression) Pipeline:** **Smooth -> Offset -> Scale**. The non-performant versions incorrectly applied the historical offset *before* scaling, which fundamentally changed the resulting signal. 3. **Resolution:** The `MainStrategy.pine` file was reverted to a monolithic state by replacing its M2 calculation block with the one from the `v36` baseline. This restored the 318k% performance, confirming that the M2 logic block is a fragile, intertwined unit. --- ## 7. Final Conclusion & Future Path 1. **The 318k% Baseline is an Emergent Property of a Specific Pipeline:** The high performance is not just a result of the indicators used, but of a specific, fragile, and non-intuitive calculation pipeline (`Smooth -> Scale -> Offset`). Any deviation from this exact sequence results in a significant performance regression. 2. **The M2 Logic Block is a "Sacred" Unit:** The M2 calculation block cannot be easily refactored, "cleaned," or modularized without breaking the fragile interactions that produce the desired performance. It must be treated as a single, monolithic unit. ### Final Recommendation The primary objective of restoring the 1D baseline has been achieved. The M2 calculation logic now resides monolithically within `MainStrategy.pine` to preserve its integrity. **Future Work on Multi-Timeframe Support:** If multi-timeframe support for the M2 indicator is ever attempted again, the following approach is recommended to avoid repeating past failures: 1. **Do Not Modify the 1D Logic:** The existing, monolithic M2 calculation block must be preserved and walled off, running *only* when `timeframe.period == "D"`. 2. **Create a Parallel Intraday Path:** For any other timeframe, a completely separate and parallel M2 calculation block should be created within an `else` block. 3. **Use Dynamic Smoothing for Intraday:** This new intraday block must use `utils.f_apply_dynamic_smoothing` to handle the `series int` lengths that result from timeframe adjustments. 4. **Re-Tune Intraday Parameters from Scratch:** The parameters for the intraday path (offsets, lengths, thresholds) cannot be assumed to be the same as the 1D path. They must be re-tuned and optimized independently, as they will be operating on a different calculation pipeline (`f_apply_dynamic_smoothing` vs `f_ma`). 5. **Establish Intelligent Defaults for New Timeframes:** When creating new parameters for shorter timeframes (e.g., `_12H`, `_8H`), the default values for any length- or offset-based parameter should be calculated to maintain a constant time duration relative to the 1D setting. For example, a 10-bar lookback on 1D should default to a 20-bar lookback on 12H (`10 * (24/12)`). This provides a much more logical starting point for tuning. Threshold-based parameters, however, should likely start at the same value as their 1D counterparts. This dual-path approach is the only way to safely experiment with intraday performance without destabilizing the known, profitable 1D baseline. --- ## 8. Critical Learning: The "Dynamic Smoothing" Trap **Date:** 2025-09-09 ### Problem An attempt was made to make all non-M2 components (RSI, ATR, Stochastics, etc.) timeframe-aware by converting their `length` parameters to `series int` types. This required replacing built-in functions like `ta.rsi` and `ta.atr` with custom-built "dynamic" equivalents (`f_dynamic_rsi`, `f_dynamic_atr`) from `LibraryUtility.pine`. ### Result This resulted in a catastrophic performance regression on the 1D baseline, dropping from 318k% to 68k%. ### Root Cause Analysis The custom `f_dynamic_*` functions, while logically sound, are not mathematically identical to TradingView's built-in `ta.*` functions. The built-in functions use highly optimized and specific smoothing algorithms (e.g., RMA/Wilder's Smoothing for RSI) that our custom EMA-based versions do not perfectly replicate. The strategy's high performance is critically dependent on the exact numerical output of the built-in functions. ### Conclusion: A Known Dead End **Any architectural change that forces the replacement of a core `ta.*` function (like `ta.rsi`, `ta.atr`, `ta.hma`) with a custom-coded dynamic equivalent should be considered a high-risk modification that is likely to break the 1D baseline.** **Future Path:** To make these components timeframe-aware, a "dual-path" approach must be used, similar to the one proposed for the M2 indicator. A conditional block (e.g., `if timeframe.period == "D"`) must be used to ensure the original, performant `ta.*` functions are *always* called on the 1D chart, while the `f_dynamic_*` versions are only used on intraday timeframes where they are strictly necessary.