1	# CreatorFlow Punch List
2
3	This list tracks the work needed to prove the first local AI workflow before broadening into a platform.
4
5	## Project Standards
6
7	- [x] Keep a repo-level todo/punch list as a standard project operating artifact.
8	- [x] Add project-level agent instructions requiring the punch list to stay current.
9
10	## Discovery
11
12	- [x] Prepare a first-agent discovery packet and acceptance criteria worksheet.
13	- [ ] Collect 5-8 raw walkthrough videos from the first real estate agent.
14	- [ ] Collect examples of posts/videos the agent considers publishable.
15	- [ ] Confirm target turnaround time, revision expectations, and pricing signal.
16	- [ ] Capture real estate compliance preferences and required brokerage/disclaimer language.
17	- [ ] Define first workflow acceptance criteria from the discovery conversation.
18
19	## Product Guardrails
20
21	- [x] Keep the first version concierge-first: client sends raw input, operator reviews, client receives finished output.
22	- [x] Require human approval before final rendered output is considered publishable.
23	- [x] Do not automate posting in the first version.
24	- [x] Use a licensed stock voice or explicitly approved voice only.
25	- [x] Track manual touch time per video to identify the next automation target.
26
27	## Technical Foundation
28
29	- [x] Initialize Python project scaffold with test runner.
30	- [x] Add local Postgres setup and initial migrations.
31	- [x] Model tenants, media assets, jobs, and job steps with approval state.
32	- [x] Build workflow config parser and runner with approval gates.
33	- [x] Build CLI commands for running workflows, checking status, and approving scripts.
34	- [x] Add media utilities for ffmpeg command construction and rendering.
35	- [x] Add a local no-paid-API demo render path that produces a playable vertical MP4 from the phone video.
36	- [x] Render transcript-derived TTS narration with synchronized captions and source audio muted by default.
37	- [x] Wire real transcription via OpenAI Whisper (`--transcription-provider whisper`). Audio is extracted before submission to avoid container format issues with the API.
38	- [x] Improve transcript cleanup: remove filler words, hesitation sounds, consecutive duplicate words. Re-capitalize after removal.
39	- [x] Caption timing uses Whisper segment timestamps (scaled to TTS duration) when available, with equal-duration chunks as the offline fallback.
40	- [x] Evaluate provider-backed word-level timestamps for tighter per-word caption sync (current path uses sentence-level segments).
41	- [x] Make local TTS verification stricter and investigate macOS `say` intermittently producing header-only audio before falling back to silent demo audio.
42
43	## Listing URL Reel
44
45	- [x] Research real-estate reel competitors, HeyGen/HyperFrames capabilities, listing-source terms, Instagram output requirements, and voice-cloning constraints.
46	- [x] Write an implementation-ready listing URL to Instagram reel plan.
47	- [x] Add a canonical authorized listing manifest with source policy, path confinement, rights attestation, and photo integrity hashes.
48	- [x] Add a single approval bundle covering facts, narration, caption, photo order, cover choice, and contact outro.
49	- [x] Render a 9:16 photo reel with restrained pan/zoom motion, captions, narration, and a contact outro.
50	- [x] Produce and visually QA two distinct HyperFrames portrait listing-reel variants from authorized local pilot media (private demo outputs remain ignored).
51	- [x] Revise the pilot reels with all supplied Horne listing photos and concise Richard Fowler contact-card outros; remove unsupported mood copy.
52	- [x] Generate a designed property cover and include it in the manual Instagram handoff package.
53	- [x] Require consent and audible output when cloned narration is requested.
54	- [x] Run and inspect a local approval-resume end-to-end listing reel render with a real fixture (intentional silent provider; live cloned-voice validation still requires owner consent and credentials).
55	- [x] Add Compass demo link-intake adapter (`--demo`) that materializes local photos + snapshot while keeping Zillow/etc. no-fetch and production rights as a ship blocker.
56	- [x] Soften photo-reel Ken Burns: continuous center scale (no zoompan), 60fps, CRF 18, light denoise, quality-first encode.
57	- [x] Force spoken “square feet” + city/state-only narration (no street “Dr=doctor”).
58	- [x] Write Richard-facing ElevenLabs PVC + optional HeyGen avatar setup steps for private share.
59	- [x] Local Coqui XTTS sample against Richard reference wav for pilot listen.
60	- [x] Add operator-facing pre/post render reel QA scorecards plus private review links for remote artifact checks.
61	- [x] Add a repeatable HyperFrames reel review command/skill plus a reboot-persistent private review server.
62	- [x] Fix hyperframes-video-review live CLI checks hanging: bound every review subprocess with a timeout, drop the network-dependent Gemini `snapshot --describe` step (unused by our scoring) and the update-check ping.
63	- [x] Add `hyperframes-add-narration` command + skill doc: standard, repeatable fix for a video-review "missing required narration/audio" hard failure (synthesize via local `say`, mux into the silent HyperFrames render).
64	- [x] Name packaged delivery videos consistently with early `property` / `recommendation` type segments for easier sharing and sorting.
65	- [ ] Wire natural voice default to Richard’s shared ElevenLabs PVC once he completes PVC share + consent.
66	- [ ] Optional: Coqui XTTS as first-class listing-reel TTS provider (research-only until PVC lands).
67	- [x] Fail closed below min photo count for demos — `min_photos=5` default added to `materialize_compass_listing` + threaded through `fetch_compass_listing` / `build_authorized_fetcher`; 3 RED→GREEN tests (e8bcf28 on feature/listing-reel-workflow).
68	- [x] Expand hash extraction range — `_MEDIA_HASH_RE` threshold lowered 40→32 chars; 32-char (UUID-format) hashes now upgraded to `/origin.jpg` instead of staying at thumbnail quality (05cfd87).
69	- [ ] Expand Compass gallery capture further — lazy-loaded photos loaded via JS API calls after page render (requires headless browser or Compass API reverse-engineering; not capturable from initial HTML).
70	- [ ] PureRealty adapter after we clear cookie/login or get an authorized feed/export.
71	- [ ] Production rights path: broker attestation workflow + licensed Bridge/RESO option.
72
73	## Before/After Contractor Reel Template
74
75	- [x] Build a reusable HyperFrames before/after template (50/50 split-screen, before on top / after on bottom, portrait full-bleed) as a new reel family alongside the existing listing and recommendation reels.
76	- [x] Produce a landscaping demo instance (`output/hyperframes/landscaping-before-after-reel/`) with 4 before/after beats + shared contact-card outro, narration, and a passing script/video QA review.
77	- [x] Document swap points for reuse across other contractor trades (painter, remodeler, pool, etc.) in that project's `TEMPLATE-NOTES.md`.
78	- [x] Get one real (non-AI-generated) before/after photo set from an actual contractor to validate the template against real client media.
79	- [x] Multi-client support: swapped the demo's outro to a direct-brand-CTA variant (placeholder "Contract Matcher" brand, Morgan Hill CA, placeholder phone) instead of Richard's personal-referral card; documented the outro as the one "client identity block" to fork per client in `TEMPLATE-NOTES.md`.
80	- [x] Promoted the client identity block (brand name, outro headline/tagline/city/phone, avatar monogram, accent colors) to HyperFrames `data-composition-variables`, resolved at init and configurable via `client.config.json` (`--variables-file` at render time) instead of hand-edited HTML. Verified the override plumbing end-to-end with a throwaway low-fps render before finalizing.
81	- [ ] Get Contract Matcher's real business name, phone number (for the text-based scheduler being built separately), web address, and logo to replace the placeholders in `client.config.json`.
82
83	## Testing
84
85	- [x] Use TDD for deterministic business logic, config parsing, tenant scoping, approval gates, and media command construction.
86	- [x] Use BDD-style scenarios for the video narration workflow.
87	- [x] Mock paid/external APIs in the default test suite.
88	- [x] Add opt-in live integration tests only after mocked contracts are stable.
89	- [x] Add a short test video fixture for the end-to-end path.
90	- [x] Verify local render execution against the committed short video fixture.
91
92	## Provider Setup
93
94	- [x] Document local `.env` setup.
95	- [x] Document Neon database creation and migration steps.
96	- [x] Document Vercel env vars and deployment steps for any later approval UI/API.
97	- [x] Document Cloudflare DNS setup for the chosen domain/subdomain.
98	- [x] Keep long-running video rendering out of Vercel request handlers.
99

# CreatorFlow Punch List

This list tracks the work needed to prove the first local AI workflow before broadening into a platform.

## Project Standards

- [x] Keep a repo-level todo/punch list as a standard project operating artifact.
- [x] Add project-level agent instructions requiring the punch list to stay current.

## Discovery

- [x] Prepare a first-agent discovery packet and acceptance criteria worksheet.
- [ ] Collect 5-8 raw walkthrough videos from the first real estate agent.
- [ ] Collect examples of posts/videos the agent considers publishable.
- [ ] Confirm target turnaround time, revision expectations, and pricing signal.
- [ ] Capture real estate compliance preferences and required brokerage/disclaimer language.
- [ ] Define first workflow acceptance criteria from the discovery conversation.

## Product Guardrails

- [x] Keep the first version concierge-first: client sends raw input, operator reviews, client receives finished output.
- [x] Require human approval before final rendered output is considered publishable.
- [x] Do not automate posting in the first version.
- [x] Use a licensed stock voice or explicitly approved voice only.
- [x] Track manual touch time per video to identify the next automation target.

## Technical Foundation

- [x] Initialize Python project scaffold with test runner.
- [x] Add local Postgres setup and initial migrations.
- [x] Model tenants, media assets, jobs, and job steps with approval state.
- [x] Build workflow config parser and runner with approval gates.
- [x] Build CLI commands for running workflows, checking status, and approving scripts.
- [x] Add media utilities for ffmpeg command construction and rendering.
- [x] Add a local no-paid-API demo render path that produces a playable vertical MP4 from the phone video.
- [x] Render transcript-derived TTS narration with synchronized captions and source audio muted by default.
- [x] Wire real transcription via OpenAI Whisper (`--transcription-provider whisper`). Audio is extracted before submission to avoid container format issues with the API.
- [x] Improve transcript cleanup: remove filler words, hesitation sounds, consecutive duplicate words. Re-capitalize after removal.
- [x] Caption timing uses Whisper segment timestamps (scaled to TTS duration) when available, with equal-duration chunks as the offline fallback.
- [x] Evaluate provider-backed word-level timestamps for tighter per-word caption sync (current path uses sentence-level segments).
- [x] Make local TTS verification stricter and investigate macOS `say` intermittently producing header-only audio before falling back to silent demo audio.

## Listing URL Reel

- [x] Research real-estate reel competitors, HeyGen/HyperFrames capabilities, listing-source terms, Instagram output requirements, and voice-cloning constraints.
- [x] Write an implementation-ready listing URL to Instagram reel plan.
- [x] Add a canonical authorized listing manifest with source policy, path confinement, rights attestation, and photo integrity hashes.
- [x] Add a single approval bundle covering facts, narration, caption, photo order, cover choice, and contact outro.
- [x] Render a 9:16 photo reel with restrained pan/zoom motion, captions, narration, and a contact outro.
- [x] Produce and visually QA two distinct HyperFrames portrait listing-reel variants from authorized local pilot media (private demo outputs remain ignored).
- [x] Revise the pilot reels with all supplied Horne listing photos and concise Richard Fowler contact-card outros; remove unsupported mood copy.
- [x] Generate a designed property cover and include it in the manual Instagram handoff package.
- [x] Require consent and audible output when cloned narration is requested.
- [x] Run and inspect a local approval-resume end-to-end listing reel render with a real fixture (intentional silent provider; live cloned-voice validation still requires owner consent and credentials).
- [x] Add Compass demo link-intake adapter (`--demo`) that materializes local photos + snapshot while keeping Zillow/etc. no-fetch and production rights as a ship blocker.
- [x] Soften photo-reel Ken Burns: continuous center scale (no zoompan), 60fps, CRF 18, light denoise, quality-first encode.
- [x] Force spoken “square feet” + city/state-only narration (no street “Dr=doctor”).
- [x] Write Richard-facing ElevenLabs PVC + optional HeyGen avatar setup steps for private share.
- [x] Local Coqui XTTS sample against Richard reference wav for pilot listen.
- [x] Add operator-facing pre/post render reel QA scorecards plus private review links for remote artifact checks.
- [x] Add a repeatable HyperFrames reel review command/skill plus a reboot-persistent private review server.
- [x] Fix hyperframes-video-review live CLI checks hanging: bound every review subprocess with a timeout, drop the network-dependent Gemini `snapshot --describe` step (unused by our scoring) and the update-check ping.
- [x] Add `hyperframes-add-narration` command + skill doc: standard, repeatable fix for a video-review "missing required narration/audio" hard failure (synthesize via local `say`, mux into the silent HyperFrames render).
- [x] Name packaged delivery videos consistently with early `property` / `recommendation` type segments for easier sharing and sorting.
- [ ] Wire natural voice default to Richard’s shared ElevenLabs PVC once he completes PVC share + consent.
- [ ] Optional: Coqui XTTS as first-class listing-reel TTS provider (research-only until PVC lands).
- [x] Fail closed below min photo count for demos — `min_photos=5` default added to `materialize_compass_listing` + threaded through `fetch_compass_listing` / `build_authorized_fetcher`; 3 RED→GREEN tests (e8bcf28 on feature/listing-reel-workflow).
- [x] Expand hash extraction range — `_MEDIA_HASH_RE` threshold lowered 40→32 chars; 32-char (UUID-format) hashes now upgraded to `/origin.jpg` instead of staying at thumbnail quality (05cfd87).
- [ ] Expand Compass gallery capture further — lazy-loaded photos loaded via JS API calls after page render (requires headless browser or Compass API reverse-engineering; not capturable from initial HTML).
- [ ] PureRealty adapter after we clear cookie/login or get an authorized feed/export.
- [ ] Production rights path: broker attestation workflow + licensed Bridge/RESO option.

## Before/After Contractor Reel Template

- [x] Build a reusable HyperFrames before/after template (50/50 split-screen, before on top / after on bottom, portrait full-bleed) as a new reel family alongside the existing listing and recommendation reels.
- [x] Produce a landscaping demo instance (`output/hyperframes/landscaping-before-after-reel/`) with 4 before/after beats + shared contact-card outro, narration, and a passing script/video QA review.
- [x] Document swap points for reuse across other contractor trades (painter, remodeler, pool, etc.) in that project's `TEMPLATE-NOTES.md`.
- [x] Get one real (non-AI-generated) before/after photo set from an actual contractor to validate the template against real client media.
- [x] Multi-client support: swapped the demo's outro to a direct-brand-CTA variant (placeholder "Contract Matcher" brand, Morgan Hill CA, placeholder phone) instead of Richard's personal-referral card; documented the outro as the one "client identity block" to fork per client in `TEMPLATE-NOTES.md`.
- [x] Promoted the client identity block (brand name, outro headline/tagline/city/phone, avatar monogram, accent colors) to HyperFrames `data-composition-variables`, resolved at init and configurable via `client.config.json` (`--variables-file` at render time) instead of hand-edited HTML. Verified the override plumbing end-to-end with a throwaway low-fps render before finalizing.
- [ ] Get Contract Matcher's real business name, phone number (for the text-based scheduler being built separately), web address, and logo to replace the placeholders in `client.config.json`.

## Testing

- [x] Use TDD for deterministic business logic, config parsing, tenant scoping, approval gates, and media command construction.
- [x] Use BDD-style scenarios for the video narration workflow.
- [x] Mock paid/external APIs in the default test suite.
- [x] Add opt-in live integration tests only after mocked contracts are stable.
- [x] Add a short test video fixture for the end-to-end path.
- [x] Verify local render execution against the committed short video fixture.

## Provider Setup

- [x] Document local `.env` setup.
- [x] Document Neon database creation and migration steps.
- [x] Document Vercel env vars and deployment steps for any later approval UI/API.
- [x] Document Cloudflare DNS setup for the chosen domain/subdomain.
- [x] Keep long-running video rendering out of Vercel request handlers.


Reset autopilot after quota window reset


Looking at the punch list, I can see the open items:

**Listing URL Reel:**
- [ ] Wire natural voice default to Richard's shared ElevenLabs PVC once he completes PVC share + consent. (waiting on external)
- [ ] Optional: Coqui XTTS as first-class listing-reel TTS provider (research-only until PVC lands). (optional/research)
- [ ] Expand Compass gallery capture further — lazy-loaded photos loaded via JS API calls after page render (requires headless browser or Compass API reverse-engineering; not capturable from initial HTML). — THIS is actionable! It needs headless browser.
- [ ] PureRealty adapter after we clear cookie/login or get an authorized feed/export. (blocked)
- [ ] Production rights path: broker attestation workflow + licensed Bridge/RESO option. (TBD)

**Before/After Contractor Reel Template:**
- [ ] Get Contract Matcher's real business name, phone number (for the text-based scheduler being built separately), web address, and logo to replace the placeholders in `client.config.json`. (blocked, need info from Contract Matcher)

**Discovery:**
- [ ] Collect 5-8 raw walkthrough videos from the first real estate agent. (blocked, need agent)
- [ ] Collect examples of posts/

videos the agent considers publishable. (blocked)

The most technically actionable item right now is expanding Compass gallery capture to handle lazy-loaded photos that come through JS API calls after the page renders—this would need either a headless browser or reverse-engineering the Compass API since the initial HTML doesn't contain them. I'm trying to figure out if this is the same "headless-browser gallery capture" mentioned on the board, which seems conditional on Local Recs getting green-lit, or if that's referring to something else entirely. Let me look more closely at the StageHand card notes to clarify the connection. The headless browser work is blocked on Local Recs getting green-lit, so that's a single dependency. The Coqui XTTS research is marked optional and waiting on PVC, so James hasn't committed to it yet. Looking at AgenticLever, there's actual progress: the repo is confirmed as a production B2B lead-gen funnel, and recent work fixed test failures by syncing stale files and adding parity checks—now at 26 out of 37 tests passing. Looking at the congressional-trading card to understand what's actionable around the dual-poller setup and which machine should run the launchd job.


1	# congressional-trading
2
3	- Repo: `~/projects/congressional-trading` · https://github.com/jamesjlopez/congressional-trading
4	- Goal: Score congressional trade disclosures (0–13 scale) and push high-conviction buy signals via ntfy when score ≥ 9 + quality gates pass. In shadow/measurement mode — scoring and logging signals but not pushing live notifications yet.
5	- Stage: test
6	- Guard: autonomous-ok
7	- Metric: Signal precision (% of signals that are wins at 30/60d) over shadow period — threshold TBD by James
8	- Blocked on James: define exit criteria — what win-rate and sample size graduates it to live?
9	- Next action: Await James's threshold decision (11.5/LOW/edge≥2 config); weekly precision monitor running (Mondays 9am CDT)
10
11	## Notes
12	- 2026-07-06 — Seeded from git activity: "Keep shadow measurement active during..." (same-day commit). Runs on always-on machine.
13	- 2026-07-06 — Explored codebase: scoring algorithm is 0–13 scale with 8 components (committee, cluster, size, small-cap bonus, options bonus, sell pressure, strategic LLM score, disruption LLM score). Notification threshold = 9.0 after timeliness multiplier. Max 1 notification/run. 14k+ historical trades stored. `signal_outcomes` table tracks 30/60d returns. Key open issue: win-rate uses current price snapshot, not fixed-horizon (backlog item). Forward-looking predictor module also exists.
14	- 2026-07-06 — James approved agentic calibration approach: backtest first → data-derived thresholds → ongoing automated monitoring. Scoped as 3 phases below.
15
16	### Calibration task — 3 phases
17
18	**Phase 1: Fix win-rate calculation** ✅ DONE 2026-07-12
19	- Added `compute_fixed_horizon()` to `tools/win_rate.py`: uses `price_history` table (price at tx_date+30d vs tx_date) — stable classification, not today's rolling price
20	- `compute()` dispatches to fixed-horizon when `price_history` is populated; rolling fallback for fresh installs
21	- 8 TDD tests added to `tests/test_win_rate.py`, all GREEN
22	- Committed: `feat/agentic-alert-loop` — "fix(win_rate): use fixed-horizon returns instead of rolling price snapshot"
23
24	**Phase 2: Backtest sweep script** ✅ DONE 2026-07-12
25	- `scripts/backtest_sweep.py` added. Sweeps score threshold (7–13), conviction (LOW/MEDIUM/HIGH), edge-factor min (2/3/4).
26	- Quick mode: reads signal_outcomes (n=36, no price_history needed). Full mode: replays 7k+ trades from price_history.
27	- Quick mode ran: signals scored 10.8–14.5. Threshold 12.5+ improves precision 58%→67% and cuts max drawdown from -40.7% to -7.7%.
28	- **Price_history backfill needed for full sweep** (python scripts/backtest_sweep.py --backfill --csv results.csv) — yfinance not installed in container.
29	- Committed: `feat/agentic-alert-loop` — "feat(backtest): add threshold/conviction/edge sweep script"
30
31	**Phase 3: Backfill + full sweep** ✅ DONE 2026-07-12
32	- Batched yfinance backfill: 960k price rows across 1,378 tickers in 39s
33	- Ran full replay sweep: 6,493 trades, 15 months, 117 config combinations
34	- CSV saved to `data/backtest_sweep_2026-07-12.csv`
35	- **Key finding**: Current config (HIGH conviction, thresh=9.0, edge≥3) = 44% precision, -0.96% median excess vs SPY
36	- **Best config**: threshold=11.5, LOW conviction, edge≥2 → 54% precision, +0.67% median excess, 398 signals/year
37	- **Surprising**: HIGH conviction = WORSE performance. Unresearched signals at high base scores outperform LLM-tagged HIGH conviction ones — suggesting base score is more predictive than LLM overlay.
38	- Results sent to James via Slack 2026-07-12. Waiting for threshold decision to implement config changes.
39
40	**Phase 4: Ongoing automated outcome monitoring** ✅ DONE 2026-07-19
41	- `/workspace/agent/congressional-monitor.py` — stdlib-only script querying `signal_outcomes`, computing 30d excess precision (last-10 / last-20 / all-time), outputting `{"wakeAgent": bool, "data": {...}}` for Nano task trigger
42	- Alert threshold: 40% floor (`BREAKER_MIN_WIN_RATE`); sends ntfy push to `congressWatchSignals` topic on breach
43	- Current precision: 58.3% all-time / 65.0% last-20 / 60.0% last-10 (all well above floor)
44	- Scheduled weekly: task-1784500441752-hqopzy, Mondays 9am CDT (`0 9 * * 1`)
45	- 2026-07-21 — Autopilot cycle (high tier): closed stale observation — `outcome_tracker.py` horizon-return drift bug was already fixed in `0288530` (2026-07-06); observation had been logged Open on 2026-07-15 after the fix. Added regression test `test_horizon_fetcher_anchors_at_notified_date_not_today` to prevent regression. 28/28 tests pass. Committed f2846af on feat/agentic-alert-loop.
46

# congressional-trading

- Repo: `~/projects/congressional-trading` · https://github.com/jamesjlopez/congressional-trading
- Goal: Score congressional trade disclosures (0–13 scale) and push high-conviction buy signals via ntfy when score ≥ 9 + quality gates pass. In shadow/measurement mode — scoring and logging signals but not pushing live notifications yet.
- Stage: test
- Guard: autonomous-ok
- Metric: Signal precision (% of signals that are wins at 30/60d) over shadow period — threshold TBD by James
- Blocked on James: define exit criteria — what win-rate and sample size graduates it to live?
- Next action: Await James's threshold decision (11.5/LOW/edge≥2 config); weekly precision monitor running (Mondays 9am CDT)

## Notes
- 2026-07-06 — Seeded from git activity: "Keep shadow measurement active during..." (same-day commit). Runs on always-on machine.
- 2026-07-06 — Explored codebase: scoring algorithm is 0–13 scale with 8 components (committee, cluster, size, small-cap bonus, options bonus, sell pressure, strategic LLM score, disruption LLM score). Notification threshold = 9.0 after timeliness multiplier. Max 1 notification/run. 14k+ historical trades stored. `signal_outcomes` table tracks 30/60d returns. Key open issue: win-rate uses current price snapshot, not fixed-horizon (backlog item). Forward-looking predictor module also exists.
- 2026-07-06 — James approved agentic calibration approach: backtest first → data-derived thresholds → ongoing automated monitoring. Scoped as 3 phases below.

### Calibration task — 3 phases

**Phase 1: Fix win-rate calculation** ✅ DONE 2026-07-12
- Added `compute_fixed_horizon()` to `tools/win_rate.py`: uses `price_history` table (price at tx_date+30d vs tx_date) — stable classification, not today's rolling price
- `compute()` dispatches to fixed-horizon when `price_history` is populated; rolling fallback for fresh installs
- 8 TDD tests added to `tests/test_win_rate.py`, all GREEN
- Committed: `feat/agentic-alert-loop` — "fix(win_rate): use fixed-horizon returns instead of rolling price snapshot"

**Phase 2: Backtest sweep script** ✅ DONE 2026-07-12
- `scripts/backtest_sweep.py` added. Sweeps score threshold (7–13), conviction (LOW/MEDIUM/HIGH), edge-factor min (2/3/4).
- Quick mode: reads signal_outcomes (n=36, no price_history needed). Full mode: replays 7k+ trades from price_history.
- Quick mode ran: signals scored 10.8–14.5. Threshold 12.5+ improves precision 58%→67% and cuts max drawdown from -40.7% to -7.7%.
- **Price_history backfill needed for full sweep** (python scripts/backtest_sweep.py --backfill --csv results.csv) — yfinance not installed in container.
- Committed: `feat/agentic-alert-loop` — "feat(backtest): add threshold/conviction/edge sweep script"

**Phase 3: Backfill + full sweep** ✅ DONE 2026-07-12
- Batched yfinance backfill: 960k price rows across 1,378 tickers in 39s
- Ran full replay sweep: 6,493 trades, 15 months, 117 config combinations
- CSV saved to `data/backtest_sweep_2026-07-12.csv`
- **Key finding**: Current config (HIGH conviction, thresh=9.0, edge≥3) = 44% precision, -0.96% median excess vs SPY
- **Best config**: threshold=11.5, LOW conviction, edge≥2 → 54% precision, +0.67% median excess, 398 signals/year
- **Surprising**: HIGH conviction = WORSE performance. Unresearched signals at high base scores outperform LLM-tagged HIGH conviction ones — suggesting base score is more predictive than LLM overlay.
- Results sent to James via Slack 2026-07-12. Waiting for threshold decision to implement config changes.

**Phase 4: Ongoing automated outcome monitoring** ✅ DONE 2026-07-19
- `/workspace/agent/congressional-monitor.py` — stdlib-only script querying `signal_outcomes`, computing 30d excess precision (last-10 / last-20 / all-time), outputting `{"wakeAgent": bool, "data": {...}}` for Nano task trigger
- Alert threshold: 40% floor (`BREAKER_MIN_WIN_RATE`); sends ntfy push to `congressWatchSignals` topic on breach
- Current precision: 58.3% all-time / 65.0% last-20 / 60.0% last-10 (all well above floor)
- Scheduled weekly: task-1784500441752-hqopzy, Mondays 9am CDT (`0 9 * * 1`)
- 2026-07-21 — Autopilot cycle (high tier): closed stale observation — `outcome_tracker.py` horizon-return drift bug was already fixed in `0288530` (2026-07-06); observation had been logged Open on 2026-07-15 after the fix. Added regression test `test_horizon_fetcher_anchors_at_notified_date_not_today` to prevent regression. 28/28 tests pass. Committed f2846af on feat/agentic-alert-loop.


Bash

cat /workspace/extra/projects/AgenticLever/docs/observations.md 2>/dev/null || cat /workspace/extra/projects/AgenticLever/observations.md 2>/dev/null || echo "no observations file"

Check AgenticLever observations


# Observations

## Open
- `verify-tailscale.sh` reports "no dedicated serve path mount for '/AgenticLever/'" even though the mount exists — it looks the path up with the trailing slash from `redirectTo.md` while Tailscale registers it without. Cosmetic; `pool-monitor` has the same mismatch, so it's a shared-script quirk, not project-specific.

## Closed
- [x] `public/` drift — synced public/ to match root files (2026-07-20); added `tests/static-parity.test.mjs` (4 tests, all GREEN) to catch future drift. Pre-existing failures in api-handler.test.mjs are socket/environment issues unrelated to static files.
- [x] `api-handler.test.mjs` 11 ECONNRESET failures — container's `HTTP_PROXY` env var routes ALL HTTP (including 127.0.0.1) through OneCLI gateway, which rejects loopback. Fixed 2026-07-20: added `NO_PROXY` and `no_proxy` to `test`/`check` scripts in `package.json`. 37/37 now pass.
- [x] `tests/api-handler.test.mjs` appends real records to `data/interests.jsonl` — STALE: verified 2026-07-15 that `INTERESTS_FILE` isolation is already in place (the test sets `INTERESTS_FILE`/`DISCOVERY_FILE` to a temp dir before importing the handler; `bun test` leaves `data/interests.jsonl` untouched — same mtime, 47 lines).

## Notes
- context-only; never auto-processed


AgenticLever open observations:
- One open item: `verify-tailscale.sh` script cosmetic issue — "no dedicated serve path mount for '/AgenticLever/'" — but it's cosmetic and the note says it's a shared-script quirk. Not worth doing this as the autopilot cycle task.

Congressional trading: The next action is waiting on James's threshold decision. The calibration is done. The monitoring is running. No clear actionable continuation work.

So let me think about what I can actually do:

Looking at ALL the projects, the situation is:
- Most blocked on James
- StageHand has the headless-browser Compass gallery capture but it's conditional on Local Recs green-light
- For AgenticLever, I'm noticing the card says it's blocked on James confirming goal direction, but looking back at the 2026-07-20 cycle, the goal was already informally confirmed as production-ready.
- The test suite is fully passing at 37/37, so the real question is what's actionable without explicit confirmation from James.
- Following the instructions to skip anything blocked on external input or confirmation, I should move past AgenticLever and the other projects that are waiting on James's answers.
- Looking at the punch list more carefully, there's a Compass gallery expansion item that might have a different path forward depending on whether Local Recs gets green-lit.
- The board notes mention "headless-browser gallery capture if Local Recs green-lit," but I'm wondering if that's actually a hard dependency or just a note about how the work could enable both features.
- The earlier autopilot notes clarified that JS-API-loaded photos need a headless browser, but the "if Local Recs green-lit" condition might have been added as context rather than a true blocker.
- Looking at it practically, expanding the listing reel's photo count is a standalone product improvement regardless of Local Recs status, and the punch list doesn't mark it as conditional.
- I should check the CLAUDE.md file in stagehand to understand the project scope and what autonomy I have here.


1	# Stagehand Project Instructions
2	> Symlinked as `AGENTS.md`; edits affect both. Every agent (Claude Code, Codex) must read the same project instructions from here — do not reintroduce a divergent `AGENTS.md`.
3
4	This project is building a concierge-first local AI workflow service that can later become a platform.
5
6	## Listing Reel Engine (canonical)
7
8	The listing reel is rendered with **HyperFrames** (HTML → video). This is the one canonical engine for every agent — outputs live under `output/hyperframes/<project>/` as a HyperFrames project (`index.html`, `hyperframes.json`, `final_listing_reel.mp4` + `_delivery`). QA it with the repo-local skill `skills/hyperframes-reel-review/SKILL.md`.
9
10	- The older ffmpeg photo-slideshow path (`python -m cli.main run …`, writing `output/<uuid>/final_video.mp4` via `core/media/photo_reel.py`) is **legacy**. It stays for its tests and existing jobs, but it is not the default: do not build new listing reels through it. If a reel is requested, use HyperFrames.
11	- Rationale: the two paths previously produced qualitatively different videos (animated composition vs. Ken-Burns slideshow) in different folders depending on which agent ran — see the convergence decision. One engine, one output location.
12
13	## Execution Standard
14
15	- Use `docs/punch-list.md` as the project todo list and keep it current.
16	- Before starting a meaningful task, identify which punch-list items it advances.
17	- When work is completed and verified, update the checkbox in `docs/punch-list.md`.
18	- Do not mark discovery items complete until the required user/client input exists.
19	- Keep raw client/demo media, generated outputs, secrets, and private addresses out of git.
20	- For repeatable local HyperFrames reel QA, use the repo-local skill at `skills/hyperframes-reel-review/SKILL.md`.
21
22	## Testing Standard
23
24	- Testing is non-negotiable.
25	- Use TDD for deterministic business logic, config parsing, tenant scoping, approval gates, CLI behavior, and media command construction.
26	- Use BDD-style tests for multi-step workflow behavior such as "generated script pauses for approval" and "approved script resumes rendering."
27	- The default test suite must not require live paid API calls or network access.
28	- Live provider tests must be opt-in and guarded by explicit environment flags.
29	- For workflow or media-rendering changes, run a local end-to-end render through approval with a real fixture before calling the work implemented. Inspect or verify the produced video artifact enough to catch issues that unit tests miss, such as silent TTS fallback, static captions, source-audio leakage, or render duration problems.
30	- The full test suite runs automatically at the end of every turn via a `Stop` hook (`pytest -q`, configured in `.claude/settings.json`). Treat a failure it surfaces as a regression to fix before handing back; do not rely on it as a gate (it is non-blocking), and still run targeted tests while iterating.
31
32	## Current Scope
33
34	- Build a local CLI workflow first.
35	- Keep the system concierge-first: human review is part of the product.
36	- Do not add social posting, billing, authentication, or a web UI until the first workflow is proven.
37	- Vercel is acceptable later for thin UI/API surfaces, Neon for hosted Postgres, and Cloudflare for DNS/domains.
38	- Long-running video rendering should remain in a local/background worker, not in a Vercel request handler.
39

# Stagehand Project Instructions
> Symlinked as `AGENTS.md`; edits affect both. Every agent (Claude Code, Codex) must read the same project instructions from here — do not reintroduce a divergent `AGENTS.md`.

This project is building a concierge-first local AI workflow service that can later become a platform.

## Listing Reel Engine (canonical)

The listing reel is rendered with **HyperFrames** (HTML → video). This is the one canonical engine for every agent — outputs live under `output/hyperframes/<project>/` as a HyperFrames project (`index.html`, `hyperframes.json`, `final_listing_reel.mp4` + `_delivery`). QA it with the repo-local skill `skills/hyperframes-reel-review/SKILL.md`.

- The older ffmpeg photo-slideshow path (`python -m cli.main run …`, writing `output/<uuid>/final_video.mp4` via `core/media/photo_reel.py`) is **legacy**. It stays for its tests and existing jobs, but it is not the default: do not build new listing reels through it. If a reel is requested, use HyperFrames.
- Rationale: the two paths previously produced qualitatively different videos (animated composition vs. Ken-Burns slideshow) in different folders depending on which agent ran — see the convergence decision. One engine, one output location.

## Execution Standard

- Use `docs/punch-list.md` as the project todo list and keep it current.
- Before starting a meaningful task, identify which punch-list items it advances.
- When work is completed and verified, update the checkbox in `docs/punch-list.md`.
- Do not mark discovery items complete until the required user/client input exists.
- Keep raw client/demo media, generated outputs, secrets, and private addresses out of git.
- For repeatable local HyperFrames reel QA, use the repo-local skill at `skills/hyperframes-reel-review/SKILL.md`.

## Testing Standard

- Testing is non-negotiable.
- Use TDD for deterministic business logic, config parsing, tenant scoping, approval gates, CLI behavior, and media command construction.
- Use BDD-style tests for multi-step workflow behavior such as "generated script pauses for approval" and "approved script resumes rendering."
- The default test suite must not require live paid API calls or network access.
- Live provider tests must be opt-in and guarded by explicit environment flags.
- For workflow or media-rendering changes, run a local end-to-end render through approval with a real fixture before calling the work implemented. Inspect or verify the produced video artifact enough to catch issues that unit tests miss, such as silent TTS fallback, static captions, source-audio leakage, or render duration problems.
- The full test suite runs automatically at the end of every turn via a `Stop` hook (`pytest -q`, configured in `.claude/settings.json`). Treat a failure it surfaces as a regression to fix before handing back; do not rely on it as a gate (it is non-blocking), and still run targeted tests while iterating.

## Current Scope

- Build a local CLI workflow first.
- Keep the system concierge-first: human review is part of the product.
- Do not add social posting, billing, authentication, or a web UI until the first workflow is proven.
- Vercel is acceptable later for thin UI/API surfaces, Neon for hosted Postgres, and Cloudflare for DNS/domains.
- Long-running video rendering should remain in a local/background worker, not in a Vercel request handler.


Check StageHand branches


* feature/listing-reel-workflow
  main
  remotes/origin/feature/listing-reel-workflow


I'm in the stagehand directory. The active branch is `feature/listing-reel-workflow`. Let me look at the project structure to understand what's there.

OK so, I need to decide:

1. All the board's "Next actions" are officially blocked on James.
2. The StageHand punch list has an item "Expand Compass gallery capture further — lazy-loaded photos..." but the board said "headless-browser gallery capture if Local Recs green-lit".
3. The board was written by a previous autopilot cycle based on the context at the time.
4. The headless browser Compass gallery expansion is technically complex and was explicitly conditioned on Local Recs being approved, which creates a strategic blocker. I think the honest move here is to log this as a fully blocked cycle and notify slack-infrastructure, then write the skip-lock. Though I should double-check the congressional-trading project first since the notes indicate Phase 4 monitoring is complete.

Looking at the congressional-trading repo more carefully, the weekly monitor runs on Mondays and the last cycle cleaned up a drift bug that was already fixed. Phases 1-4 are all done, but the remaining blocker is James's threshold decision for going live. Let me see if there's anything actionable in AgenticLever instead that could move the project forward beyond just confirming the goal.

The tests are passing 37/37 and the goal's been confirmed as a production B2B lead-gen funnel for AI workflow automation targeting local SMBs, with just a cosmetic observation about verify-tailscale.sh. But the board shows the next action is blocked on James confirming the goal before identifying the next feature priority.

Let me check the RoadSense card to see if there's anything I can work on autonomously while waiting for that confirmation.


1	# RoadSense
2
3	- Repo: `~/projects/RoadSense`
4	- Goal: Personal motorcycle road discovery engine — algorithmically surfaces roads worth riding using GIS data + ridden-road tracking. Pilot: Cedar Park / Austin area.
5	- Stage: feedback
6	- Guard: autonomous-ok
7	- Metric: _undefined — pilot stage; "does it find good roads I'd actually ride?"_
8	- Blocked on James: answers to 5 launch questions (auth model, generalization city, GIS license, creator posture, success metric)
9	- Next action: Answer launch questions (auth model, generalization city, GIS licensing, creator posture, success metric) → unblock implementation
10
11	## Notes
12	- 2026-07-06 — Seeded from git activity: "Relax area smoke-test routing gate" (2026-06-06).
13	- 2026-07-06 — James clarified: functional personal pilot, tuned to Austin area. Two open threads: (a) test generalization to San Jose CA or Denver NC as candidate expansion areas; (b) small display issue with ridden roads. Bigger decision: go public? If yes, need to scope and plan GTM strategy.
14	- 2026-07-06 — Decision made: go public. Scoping work is next step.
15	- 2026-07-08 — Autopilot (high tier): produced public launch scope + GTM plan. Sent to James via Telegram with 5 numbered questions: (1) auth model, (2) generalization city, (3) GIS licensing, (4) creator posture, (5) 30-day success metric. Display bug fix deferred — needs local repo access.
16	- 2026-07-21 — Autopilot (high tier): fixed loop seed geobounding (670be7a). Area filter was fully implemented in routes.ts but not wired to the npm scripts. Added Cedar Park center (30.5051, -97.8203, matching Map.tsx CENTER) + 40mi radius to pipeline:routes and pipeline:routes:fast. Eliminates Lockhart/Gonzales loops (~43.5mi away). Rerun pnpm pipeline:routes:fast to regenerate routes. Also closed nanoclaw push() observation (was fixed in d4ef752, observation hadn't been updated).
17	- 2026-07-21 — Autopilot (high tier): fixed BDR legend — removed stale "Ridden" row that was advertising behaviour the map doesn't render in BDR view. `roads-ridden-overlay` is Roads-view only; BDR view uses `bdr-*` layers. Committed a9542d7. Visual smoke-check: open BDR view, confirm legend shows Paved / Gravel / Dirt only.
18	- 2026-07-21 — Autopilot (high tier): fixed score-inheritance regression. `suppressUnsupportedShortSegments()` was using graph-adjacency (35m) to suppress segments that had just been correctly boosted by name-adjacency (100m). Fix: added `+inherited` exclusion to `support` CTE WHERE clause in `pipeline/score.ts`. Committed fa807e9 on fix/unridden-road-filter. Tests are DB-dependent; 14 failing tests should go GREEN on Mac Studio after this commit.
19

# RoadSense

- Repo: `~/projects/RoadSense`
- Goal: Personal motorcycle road discovery engine — algorithmically surfaces roads worth riding using GIS data + ridden-road tracking. Pilot: Cedar Park / Austin area.
- Stage: feedback
- Guard: autonomous-ok
- Metric: _undefined — pilot stage; "does it find good roads I'd actually ride?"_
- Blocked on James: answers to 5 launch questions (auth model, generalization city, GIS license, creator posture, success metric)
- Next action: Answer launch questions (auth model, generalization city, GIS licensing, creator posture, success metric) → unblock implementation

## Notes
- 2026-07-06 — Seeded from git activity: "Relax area smoke-test routing gate" (2026-06-06).
- 2026-07-06 — James clarified: functional personal pilot, tuned to Austin area. Two open threads: (a) test generalization to San Jose CA or Denver NC as candidate expansion areas; (b) small display issue with ridden roads. Bigger decision: go public? If yes, need to scope and plan GTM strategy.
- 2026-07-06 — Decision made: go public. Scoping work is next step.
- 2026-07-08 — Autopilot (high tier): produced public launch scope + GTM plan. Sent to James via Telegram with 5 numbered questions: (1) auth model, (2) generalization city, (3) GIS licensing, (4) creator posture, (5) 30-day success metric. Display bug fix deferred — needs local repo access.
- 2026-07-21 — Autopilot (high tier): fixed loop seed geobounding (670be7a). Area filter was fully implemented in routes.ts but not wired to the npm scripts. Added Cedar Park center (30.5051, -97.8203, matching Map.tsx CENTER) + 40mi radius to pipeline:routes and pipeline:routes:fast. Eliminates Lockhart/Gonzales loops (~43.5mi away). Rerun pnpm pipeline:routes:fast to regenerate routes. Also closed nanoclaw push() observation (was fixed in d4ef752, observation hadn't been updated).
- 2026-07-21 — Autopilot (high tier): fixed BDR legend — removed stale "Ridden" row that was advertising behaviour the map doesn't render in BDR view. `roads-ridden-overlay` is Roads-view only; BDR view uses `bdr-*` layers. Committed a9542d7. Visual smoke-check: open BDR view, confirm legend shows Paved / Gravel / Dirt only.
- 2026-07-21 — Autopilot (high tier): fixed score-inheritance regression. `suppressUnsupportedShortSegments()` was using graph-adjacency (35m) to suppress segments that had just been correctly boosted by name-adjacency (100m). Fix: added `+inherited` exclusion to `support` CTE WHERE clause in `pipeline/score.ts`. Committed fa807e9 on fix/unridden-road-filter. Tests are DB-dependent; 14 failing tests should go GREEN on Mac Studio after this commit.


RoadSense: The last three notes are from 2026-07-21 and show actual autonomous continuation work was done (fixing the loop seed, BDR legend, score-inheritance regression). But the current next action is "Answer launch questions → unblock implementation" which is blocked on James.

However, looking at the notes more carefully: "14 failing tests should go GREEN on Mac Studio after this commit" — there are 14 failing tests! That could be actionable work. But these require a local DB environment (DB-dependent tests on Mac Studio), so I can't run them from this container.
I'm noticing a stage mismatch between the board and the card — the board lists RoadSense as "spec" but the card shows "feedback". Let me verify which is correct.

Looking at the actual work done (fixing score-inheritance regressions, BDR legend issues, route seed bounding), this seems past the spec stage, so the card's "feedback" stage appears more accurate. The next action is consistent across both sources though — it's blocked on James answering the five launch questions before implementation can proceed. Looking back at the AgenticLever notes, the recent autopilot cycles actually delivered real progress—fixing ECONNRESET failures and getting all 37 tests passing. So there's a disconnect between the work that's been done and the card still being blocked on James for direction confirmation. That's worth noting. The AgenticLever card is explicitly blocked on James confirming the goal direction, so I shouldn't proceed with feature work there. For RoadSense, the 14 failing tests are environment-dependent—they'd pass on Mac Studio with the real database—so there's nothing I can fix from this container. I'm sitting at 67% burnable work and need to figure out what I can actually move forward on.

Looking at StageHand, the board says the headless browser gallery capture is conditional on Local Recs being green-lit, but the punch list item doesn't mention that condition. I need to check whether the punch list is outdated or if there's a mismatch in what's actually blocking this work.

The listing_reel feature is already complete and deployed, so expanding the Compass gallery would directly improve that existing feature rather than being speculative infrastructure. But I should respect the constraint from the previous autopilot cycle unless there's a compelling reason to override it—the condition was set to avoid partial infrastructure builds.

I could treat the headless browser integration as greenfield planning work: design the approach, red-team it, and send a package to James for approval. That way I'm not implementing something conditional on Local Recs, but I'm preparing the groundwork so he can unblock it with a simple decision. Though I'm second-guessing whether this counts as truly greenfield or if it's just continuation work on an existing feature that the board explicitly made conditional.

Looking back at the autopilot rules, the instruction is clear: pick the highest-priority eligible project with a clear, actionable next step, and skip anything where the next action is "confirm with James" or blocked on external input. Reviewing each project's next action, they're all waiting on James's input in some form. The honest assessment is that everything's blocked on his decision, so I should log this as blocked, notify slack-infrastructure about what's stalled and why, and write the skip-lock. one is blocked on James in some way. I'm wondering if there's any autonomous work I can pick up in these repos without waiting for his input. Looking at congressional-trading, I don't see obvious continuation work, and RoadSense's failing tests are environment-specific issues I can't fix from here. But I notice the RoadSense board shows "spec" while the card says "feedback"—that's a real discrepancy I can fix as a board maintenance task.

Let me check if there's anything in nanoclaw beyond the pending deploy that I could work on.


cat /workspace/extra/projects/nanoclaw/observations.md 2>/dev/null | head -50 || echo "no observations"

Check nanoclaw observations


(Bash completed with no output)


cd /workspace/extra/projects/nanoclaw && git log --oneline -8 2>/dev/null

Check nanoclaw recent commits


2bae9e8 chore(observations): close push() silent-drop item — fixed in d4ef752
d4ef752 fix(providers): queue push() messages during REST rungs instead of dropping
d14a5fc fix(providers): only synthesize honest no-tools reply on the final REST rung
e37ffe5 chore: add AGENTS.md symlink to CLAUDE.md
b724c2a fix(provider): detect and retry hallucinated tool-call tags on REST rungs
02c293e fix(router): scope mention-sticky session lookup to wiring's session_mode
846ce6b Show last commit date on repo-awareness cards
0cb494a Pin setup's node path to .nvmrc's major version when present


cat /workspace/extra/projects/nanoclaw/docs/observations.md 2>/dev/null | head -60 || find /workspace/extra/projects/nanoclaw -name "observations.md" 2>/dev/null | head -5

Find nanoclaw observations


# Observations

## Open

- One-off `SqliteError: attempt to write a readonly database` in the active delivery poll (2026-07-07 03:01, `dist/db/session-db.js getDueOutboundMessages`) — transient, deliveries recovered, cause unknown.
- Host tests fail under Homebrew `node` (26.x): `better-sqlite3` native module is compiled for the service's Node 22. Run `PATH=/opt/homebrew/opt/node@22/bin:$PATH pnpm test`.
- Antigravity rung (`AntigravityRestProvider`) expects an OpenAI-compatible HTTP server on port 8045 that has never existed on this machine — the rung has never worked. `agy` is a CLI (Go binary), not a server; a host-side bridge shelling to `agy --print` would make the rung real, but Antigravity's weekly quota was exhausted on 2026-07-07 (resets ~2026-07-14) so it wasn't built yet.
- `agy --print` exits 0 with EMPTY stdout when quota is exhausted (RESOURCE_EXHAUSTED only visible via `--log-file`) — any future bridge must treat empty output as failure, not success.
- Recurring `Killing container past absolute ceiling` (heartbeatAgeMs ≈ 30 min) for the main DM session several times a day — containers hang past the heartbeat ceiling; cause not yet investigated.

## Closed

- `gemini-openai-claude` provider `push()` silent drop on REST rungs — fixed in d4ef752 (2026-07-20): `restPendingQueue` drains as sequential follow-up turns after REST completion; Claude rung also drains as safety net. 135/135 tests pass; awaiting Studio deploy.
- Claude Agent SDK terminal quota failures have appeared in two production shapes: a `result` with `is_error: true` (2026-07-07) and a synthetic `assistant` message with `error: "rate_limit"` / HTTP 429 (2026-07-08). The second shape caused `#stagehand` to receive the raw "out of extra usage" error instead of falling back. Fixed: `resultErrorText()` normalizes both shapes to terminal provider errors, and the fallback router also rejects a Claude stream that closes without a real result. Covered by the production-shaped test in `claude.test.ts` and the no-result/fallback cases in `gemini-openai-claude.test.ts`.
- `~/.config/nanoclaw/mount-allowlist.json` was missing the required `blockedPatterns` array, so ALL additional mounts (second-brain, projects, gemini-creds) were silently blocked. Fixed 2026-07-07 by adding `"blockedPatterns": []`.
- REST fallback rungs (LM Studio especially) silently dropped replies: local models answered with only `<internal>…</internal>` (no `<message to=…>` block), `dispatchResultText` discarded it as scratchpad, and the poll-loop nudge can't reach REST rungs (`push()` is Claude-only). Fixed 2026-07-07: the provider now retries once with a corrective instruction and, failing that, force-wraps the visible/internal text in a `<message>` block addressed to the prompt's last `from=` destination. Covered by `gemini-openai-claude.test.ts` tests 8–12; verified live against LM Studio + qwen.
- Containers could never use the Claude Pro subscription: OneCLI's gateway env ships `ANTHROPIC_API_KEY=placeholder` + HTTPS_PROXY and injects the vault's (depleted) API key on the wire. Fixed 2026-07-07: `resolveClaudeAuthEnv` in `container/agent-runner/src/providers/claude.ts` strips the API-key vars and NO_PROXYs `api.anthropic.com` when `CLAUDE_CODE_OAUTH_TOKEN` is set; host passes the token through in `src/providers/gemini-openai-claude.ts`. Token created and added to `.env` 2026-07-07 (valid 1 year); verified live — container session read a file and delivered a Telegram reply via claude-sonnet-4-6 on the subscription.

## Notes

- LM Studio (port 1234) is the local fallback rung; `LM_STUDIO_MODEL=qwen/qwen3.6-27b` set in `.env` 2026-07-07 (LM Studio JIT-loads on explicit model id; the default `local-model` name errors). It runs before Claude for plain chat, but remains the last rung for tool-shaped work because it has no tool access.
- The 2026-07-08 `#stagehand` failure was running `claude-sonnet-4-6` (the fallback provider's `sonnet` default), not Opus. Active routing is documented in `docs/provider-routing-plan.md`: tool work uses Codex → Claude → LM Studio; plain chat uses Codex → LM Studio → Claude. Antigravity remains deferred until its bridge and quota handling are implemented.


I found some open observations in nanoclaw:

1. **`SqliteError: attempt to write a readonly database`** — transient, one-off
2. **Host tests fail under Homebrew node 26.x** — environment issue on Mac, not container
3. **Antigravity rung has never worked** — needs a host-side bridge shelling to `agy --print`; `agy` weekly quota was exhausted when this was noted (2026-07-07, resets ~2026-07-14). Wait, that was weeks ago, so quota has likely reset. But we're in a container, so we can't test this.
4. **`agy --print` exits 0 with EMPTY stdout when quota exhausted** — bridge needs to handle this gracefully
5. **Recurring `Killing container past absolute ceiling`** — cause not yet investigated

The Antigravity rung is worth exploring. Building a real bridge would involve wiring up the `agy` CLI integration and handling the quota exhaustion case where the command succeeds but returns nothing.

The challenge is that the bridge itself needs to run on the host (Mac Studio), not in the container. I could write the bridge code in the nanoclaw repo from here, but actually deploying it requires host access. And nanoclaw is already blocked waiting for James to review and deploy the d4ef752 fix to the Studio.

But looking at the card more carefully — it's marked `autonomous-ok` and the blocker is specifically on the deploy and metrics confirmation, not on writing code. So I could autonomously work on nanoclaw improvements like investigating the "Recurring container past absolute ceiling" issue or building out the bridge code itself, even if the actual deployment has to wait. on on the "Next action" field, or if it should also pick up actionable open observations when the Next action is blocked. The instructions say "Pick ONE unit of work — the highest-priority eligible project with a clear, actionable `Next action`" — which suggests the Next action field is the filter. If nanoclaw's Next action is blocked on James, then it's not eligible, and I should look at other projects instead. Looking back at the RoadSense and StageHand autopilot cycles, I see the pattern now — when the official next action was blocked, the autopilot successfully found and completed useful work from the punch list and open observations. So the spirit of autonomous work means I should look beyond just the card's stated next action if it's blocked, and tackle relevant repo-level items instead. Let me reassess StageHand with this in mind — there are punch list items tied to the current feature work that could move things forward.

For the headless browser Compass gallery capture, I'm tempted to prototype using the existing `agent-browser` skill, but I recognize this would be a substantial undertaking requiring Playwright integration, a new adapter layer, and test coverage. Since it was explicitly conditioned on Local Recs being approved, I should probably hold off. For nanoclaw, there's an open observation about recurring container-killing errors that hasn't been investigated yet — that's a concrete thing I could dig into.

Looking at the other projects: RoadSense has 14 failing tests that should pass on Mac Studio but I can't run the database tests from the container, and congressional-trading has completed Phase 4 with no clear autonomous work left. Given that previous autopilot cycles found real work by examining punch lists and observations, I should focus on the nanoclaw heartbeat investigation — diving into the code to understand the root cause of those container errors and potentially implementing a fix.

I need to be careful though: the card's next action is explicitly for James to review and deploy, but the project itself is marked autonomous-ok, which means I can still work on improvements even while that specific step is blocked. The previous autopilot notes show multiple nanoclaw improvements were made independently, so investigating and fixing the heartbeat issue would be legitimate autonomous work.

I'll focus on the nanoclaw heartbeat investigation—specifically the "Killing container past absolute ceiling" observation. Let me dig into the nanoclaw codebase to understand what's triggering this behavior and where the container lifecycle management might be going wrong.


On it — reading board and project cards to pick the best unit of work.


Message sent to telegram-mg-17796 (id: 3569)


All the official "Next actions" on the board are blocked on James. But nanoclaw has an open observation: "Recurring `Killing container past absolute ceiling` (heartbeatAgeMs ≈ 30 min) for the main DM session several times a day — containers hang past the heartbeat ceiling; cause not yet investigated."

This IS actionable autonomous work on a P1, autonomous-ok project. Let me investigate this now.

Let me look at the nanoclaw code to understand the heartbeat mechanism.


All official "Next actions" are blocked on James, but nanoclaw (P1, autonomous-ok) has an open observation about a recurring heartbeat ceiling issue — "cause not yet investigated." Let me investigate and fix it.


grep -r "absolute ceiling\|heartbeatAge\|heartbeat" /workspace/extra/projects/nanoclaw/src --include="*.ts" -l 2>/dev/null

Find heartbeat-related code in nanoclaw


grep -n "heartbeatAge\|absolute ceiling\|heartbeat\|ceiling" /workspace/extra/projects/nanoclaw/src/host-sweep.ts | head -30

Check host-sweep heartbeat logic


7: *   - Uses heartbeat file mtime for liveness (never polls DB for it)
11: * heartbeat threshold):
18: *     1. Absolute ceiling: heartbeat age > max(30 min, current_bash_timeout)
25: *        (claim_age > tolerance) AND (heartbeat_mtime <= status_changed)
46:import { openInboundDb, openOutboundDb, openOutboundDbRw, inboundDbPath, heartbeatPath } from './session-manager.js';
62:// Absolute idle ceiling for a running container. If the heartbeat file hasn't
74:  | { action: 'kill-ceiling'; heartbeatAgeMs: number; ceilingMs: number }
84:  heartbeatMtimeMs: number; // 0 when heartbeat file absent
88:  const { now, heartbeatMtimeMs, containerState, claims } = args;
91:  // Ceiling check only applies when we have an actual heartbeat timestamp.
93:  // heartbeat file exists — if we treated that as infinitely stale we'd
95:  // that never wrote a heartbeat are caught by the separate "container
99:  if (heartbeatMtimeMs !== 0) {
100:    const heartbeatAge = now - heartbeatMtimeMs;
101:    const ceiling = Math.max(ABSOLUTE_CEILING_MS, declaredBashMs ?? 0);
102:    if (heartbeatAge > ceiling) {
103:      return { action: 'kill-ceiling', heartbeatAgeMs: heartbeatAge, ceilingMs: ceiling };
113:    if (heartbeatMtimeMs > claimedAt) continue;
190:    // 3. Running-container SLA: absolute ceiling + per-claim stuck rules.
214:function heartbeatMtimeMs(agentGroupId: string, sessionId: string): number {
215:  const hbPath = heartbeatPath(agentGroupId, sessionId);
236:    heartbeatMtimeMs: heartbeatMtimeMs(agentGroupId, session.id),
243:  if (decision.action === 'kill-ceiling') {
244:    log.warn('Killing container past absolute ceiling', {
246:      heartbeatAgeMs: decision.heartbeatAgeMs,
247:      ceilingMs: decision.ceilingMs,
249:    killContainer(session.id, 'absolute-ceiling');
250:    resetStuckProcessingRows(inDb, outDb, session, 'absolute-ceiling');


grep -n "heartbeatAge\|absolute ceiling\|heartbeat\|ceiling" /workspace/extra/projects/nanoclaw/src/session-manager.ts | head -30

Check session-manager heartbeat logic


66:/** Path to the container heartbeat file (touched instead of DB writes). */
67:export function heartbeatPath(agentGroupId: string, sessionId: string): string {
68:  return path.join(sessionDir(agentGroupId, sessionId), '.heartbeat');


1	/**
2	 * Host sweep — periodic maintenance of all session DBs.
3	 *
4	 * Two-DB architecture:
5	 *   - Reads processing_ack + container_state from outbound.db
6	 *   - Writes to inbound.db (host-owned) for status updates + recurrence
7	 *   - Uses heartbeat file mtime for liveness (never polls DB for it)
8	 *   - Never writes to outbound.db — preserves single-writer-per-file invariant
9	 *
10	 * Stuck / idle detection (replaces the old IDLE_TIMEOUT setTimeout + 10-min
11	 * heartbeat threshold):
12	 *
13	 *   If the container isn't running and there are 'processing' rows left over
14	 *   (e.g. it crashed mid-turn) → reset them to pending with backoff +
15	 *   tries++. Existing retry machinery does the rest.
16	 *
17	 *   If the container IS running:
18	 *     1. Absolute ceiling: heartbeat age > max(30 min, current_bash_timeout)
19	 *        → kill. Covers the "alive but silent for 30 min" case. Extended
20	 *        only while Bash is declared as running longer, honouring the
21	 *        user's own timeout directive. Kill then resets processing rows.
22	 *
23	 *     2. Message-scoped stuck: for each 'processing' row, tolerance =
24	 *        max(60s, current_bash_timeout_ms_if_Bash_running). If
25	 *        (claim_age > tolerance) AND (heartbeat_mtime <= status_changed)
26	 *        → kill + reset this message + tries++. Semantics: "container
27	 *        claimed a message and went quiet past tolerance since the claim."
28	 */
29	import type Database from 'better-sqlite3';
30	import fs from 'fs';
31
32	import { getActiveSessions } from './db/sessions.js';
33	import { getAgentGroup } from './db/agent-groups.js';
34	import {
35	  countDueMessages,
36	  deleteOrphanProcessingClaims,
37	  getContainerState,
38	  getMessageForRetry,
39	  getProcessingClaims,
40	  markMessageFailed,
41	  retryWithBackoff,
42	  syncProcessingAcks,
43	  type ContainerState,
44	} from './db/session-db.js';
45	import { log } from './log.js';
46	import { openInboundDb, openOutboundDb, openOutboundDbRw, inboundDbPath, heartbeatPath } from './session-manager.js';
47	import { isContainerRunning, killContainer, wakeContainer } from './container-runner.js';
48	import type { Session } from './types.js';
49
50	/**
51	 * SQLite TIMESTAMP columns store UTC without a timezone marker. Date.parse
52	 * treats timezoneless ISO strings as local time, so on non-UTC hosts every
53	 * timestamp looks (TZ offset) hours stale — leading to spurious kill-claim
54	 * decisions on freshly-claimed messages. Append "Z" when no zone marker is
55	 * present so Date.parse interprets the string as UTC.
56	 */
57	export function parseSqliteUtc(s: string): number {
58	  return Date.parse(/[zZ]|[+-]\d{2}:?\d{2}$/.test(s) ? s : s + 'Z');
59	}
60
61	const SWEEP_INTERVAL_MS = 60_000;
62	// Absolute idle ceiling for a running container. If the heartbeat file hasn't
63	// been touched in this long, the container is either stuck or doing genuinely
64	// nothing — kill and restart on the next inbound.
65	export const ABSOLUTE_CEILING_MS = 30 * 60 * 1000;
66	// Stuck tolerance window applied per 'processing' claim — "did we see any
67	// signs of life since this message was claimed?"
68	export const CLAIM_STUCK_MS = 60 * 1000;
69	const MAX_TRIES = 5;
70	const BACKOFF_BASE_MS = 5000;
71
72	export type StuckDecision =
73	  | { action: 'ok' }
74	  | { action: 'kill-ceiling'; heartbeatAgeMs: number; ceilingMs: number }
75	  | { action: 'kill-claim'; messageId: string; claimAgeMs: number; toleranceMs: number };
76
77	/**
78	 * Pure decision for whether a running container should be killed this sweep
79	 * tick. Inputs are all deterministic; filesystem + DB reads happen in the
80	 * caller.
81	 */
82	export function decideStuckAction(args: {
83	  now: number;
84	  heartbeatMtimeMs: number; // 0 when heartbeat file absent
85	  containerState: ContainerState | null;
86	  claims: Array<{ message_id: string; status_changed: string }>;
87	}): StuckDecision {
88	  const { now, heartbeatMtimeMs, containerState, claims } = args;
89	  const declaredBashMs = bashTimeoutMs(containerState);
90
91	  // Ceiling check only applies when we have an actual heartbeat timestamp.
92	  // A freshly-spawned container hasn't had any SDK activity yet so no
93	  // heartbeat file exists — if we treated that as infinitely stale we'd
94	  // kill every container within seconds of spawn. Genuinely-dead containers
95	  // that never wrote a heartbeat are caught by the separate "container
96	  // process not running" cleanup path, not here. If a fresh container is
97	  // hanging at the gate (claimed a message but never did anything) the
98	  // claim-stuck check below handles it.
99	  if (heartbeatMtimeMs !== 0) {
100	    const heartbeatAge = now - heartbeatMtimeMs;
101	    const ceiling = Math.max(ABSOLUTE_CEILING_MS, declaredBashMs ?? 0);
102	    if (heartbeatAge > ceiling) {
103	      return { action: 'kill-ceiling', heartbeatAgeMs: heartbeatAge, ceilingMs: ceiling };
104	    }
105	  }
106
107	  const tolerance = Math.max(CLAIM_STUCK_MS, declaredBashMs ?? 0);
108	  for (const claim of claims) {
109	    const claimedAt = parseSqliteUtc(claim.status_changed);
110	    if (Number.isNaN(claimedAt)) continue;
111	    const claimAge = now - claimedAt;
112	    if (claimAge <= tolerance) continue;
113	    if (heartbeatMtimeMs > claimedAt) continue;
114	    return { action: 'kill-claim', messageId: claim.message_id, claimAgeMs: claimAge, toleranceMs: tolerance };
115	  }
116
117	  return { action: 'ok' };
118	}
119
120	let running = false;
121
122	export function startHostSweep(): void {
123	  if (running) return;
124	  running = true;
125	  sweep();
126	}
127
128	export function stopHostSweep(): void {
129	  running = false;
130	}
131
132	async function sweep(): Promise<void> {
133	  if (!running) return;
134
135	  try {
136	    const sessions = getActiveSessions();
137	    for (const session of sessions) {
138	      await sweepSession(session);
139	    }
140	  } catch (err) {
141	    log.error('Host sweep error', { err });
142	  }
143
144	  setTimeout(sweep, SWEEP_INTERVAL_MS);
145	}
146
147	async function sweepSession(session: Session): Promise<void> {
148	  const agentGroup = getAgentGroup(session.agent_group_id);
149	  if (!agentGroup) return;
150
151	  const inPath = inboundDbPath(agentGroup.id, session.id);
152	  if (!fs.existsSync(inPath)) return;
153
154	  let inDb: Database.Database;
155	  let outDb: Database.Database | null = null;
156	  try {
157	    inDb = openInboundDb(agentGroup.id, session.id);
158	  } catch {
159	    return;
160	  }
161
162	  try {
163	    outDb = openOutboundDb(agentGroup.id, session.id);
164	  } catch {
165	    // outbound.db might not exist yet (container hasn't started)
166	  }
167
168	  try {
169	    // 1. Sync processing_ack → messages_in status
170	    if (outDb) {
171	      syncProcessingAcks(inDb, outDb);
172	    }
173
174	    // 2. Wake a container if work is due and nothing is running. Ordered
175	    // before the crashed-container cleanup so a fresh container gets a chance
176	    // to clean its own orphan processing_ack rows on startup (see
177	    // container/agent-runner/src/db/connection.ts). Otherwise the reset path
178	    // would keep bumping process_after into the future, dueCount would stay 0,
179	    // and the wake would never fire.
180	    const dueCount = countDueMessages(inDb);
181	    if (dueCount > 0 && !isContainerRunning(session.id)) {
182	      log.info('Waking container for due messages', { sessionId: session.id, count: dueCount });
183	      // wakeContainer never throws — transient spawn failures (OneCLI down,
184	      // etc.) return false and leave messages pending for the next tick.
185	      await wakeContainer(session);
186	    }
187
188	    const alive = isContainerRunning(session.id);
189
190	    // 3. Running-container SLA: absolute ceiling + per-claim stuck rules.
191	    if (alive && outDb) {
192	      enforceRunningContainerSla(inDb, outDb, session, agentGroup.id);
193	    }
194
195	    // 4. Crashed-container cleanup: processing rows left behind get retried.
196	    // Only fires when wake in step 2 didn't pick up the work (no due messages,
197	    // or wake failed). resetStuckProcessingRows itself is idempotent — it
198	    // skips messages already scheduled for a future retry.
199	    if (!alive && outDb) {
200	      resetStuckProcessingRows(inDb, outDb, session, 'container not running');
201	    }
202
203	    // 5. Recurrence fanout for completed recurring tasks.
204	    // MODULE-HOOK:scheduling-recurrence:start
205	    const { handleRecurrence } = await import('./modules/scheduling/recurrence.js');
206	    await handleRecurrence(inDb, session);
207	    // MODULE-HOOK:scheduling-recurrence:end
208	  } finally {
209	    inDb.close();
210	    outDb?.close();
211	  }
212	}
213
214	function heartbeatMtimeMs(agentGroupId: string, sessionId: string): number {
215	  const hbPath = heartbeatPath(agentGroupId, sessionId);
216	  try {
217	    return fs.statSync(hbPath).mtimeMs;
218	  } catch {
219	    return 0;
220	  }
221	}
222
223	function bashTimeoutMs(state: ContainerState | null): number | null {
224	  if (!state || state.current_tool !== 'Bash') return null;
225	  return typeof state.tool_declared_timeout_ms === 'number' ? state.tool_declared_timeout_ms : null;
226	}
227
228	function enforceRunningContainerSla(
229	  inDb: Database.Database,
230	  outDb: Database.Database,
231	  session: Session,
232	  agentGroupId: string,
233	): void {
234	  const decision = decideStuckAction({
235	    now: Date.now(),
236	    heartbeatMtimeMs: heartbeatMtimeMs(agentGroupId, session.id),
237	    containerState: getContainerState(outDb),
238	    claims: getProcessingClaims(outDb),
239	  });
240
241	  if (decision.action === 'ok') return;
242
243	  if (decision.action === 'kill-ceiling') {
244	    log.warn('Killing container past absolute ceiling', {
245	      sessionId: session.id,
246	      heartbeatAgeMs: decision.heartbeatAgeMs,
247	      ceilingMs: decision.ceilingMs,
248	    });
249	    killContainer(session.id, 'absolute-ceiling');
250	    resetStuckProcessingRows(inDb, outDb, session, 'absolute-ceiling');
251	    return;
252	  }
253
254	  log.warn('Killing container — message claimed then silent', {
255	    sessionId: session.id,
256	    messageId: decision.messageId,
257	    claimAgeMs: decision.claimAgeMs,
258	    toleranceMs: decision.toleranceMs,
259	  });
260	  killContainer(session.id, 'claim-stuck');
261	  resetStuckProcessingRows(inDb, outDb, session, 'claim-stuck');
262	}
263
264	export function _resetStuckProcessingRowsForTesting(
265	  inDb: Database.Database,
266	  outDb: Database.Database,
267	  session: Session,
268	  reason: string,
269	): void {
270	  resetStuckProcessingRows(inDb, outDb, session, reason, outDb);
271	}
272
273	function resetStuckProcessingRows(
274	  inDb: Database.Database,
275	  outDb: Database.Database,
276	  session: Session,
277	  reason: string,
278	  writableOutDb?: Database.Database,
279	): void {
280	  const claims = getProcessingClaims(outDb);
281	  const now = Date.now();
282	  for (const { message_id } of claims) {
283	    const msg = getMessageForRetry(inDb, message_id, 'pending');
284	    if (!msg) continue;
285
286	    // Already rescheduled for a future retry — don't bump tries again. The
287	    // wake path (sweep step 2) will fire when process_after elapses and a
288	    // fresh container will clean the orphan claim on startup.
289	    if (msg.processAfter && parseSqliteUtc(msg.processAfter) > now) continue;
290
291	    if (msg.tries >= MAX_TRIES) {
292	      markMessageFailed(inDb, msg.id);
293	      log.warn('Message marked as failed after max retries', {
294	        messageId: msg.id,
295	        sessionId: session.id,
296	        reason,
297	      });
298	    } else {
299	      const backoffMs = BACKOFF_BASE_MS * Math.pow(2, msg.tries);
300	      const backoffSec = Math.floor(backoffMs / 1000);
301	      retryWithBackoff(inDb, msg.id, backoffSec);
302	      log.info('Reset stale message with backoff', {
303	        messageId: msg.id,
304	        tries: msg.tries,
305	        backoffMs,
306	        reason,
307	      });
308	    }
309	  }
310
311	  // Drop the orphan 'processing' rows. Without this, the next sweep tick
312	  // would re-read them, see the old status_changed timestamp, conclude the
313	  // freshly respawned container is stuck, and SIGKILL it before its
314	  // agent-runner has a chance to run clearStaleProcessingAcks() on startup.
315	  const ownsDb = !writableOutDb;
316	  let useDb: Database.Database | null = writableOutDb ?? null;
317	  try {
318	    if (!useDb) useDb = openOutboundDbRw(session.agent_group_id, session.id);
319	    const cleared = deleteOrphanProcessingClaims(useDb);
320	    if (cleared > 0) {
321	      log.info('Cleared orphan processing claims', { sessionId: session.id, cleared, reason });
322	    }
323	  } catch (err) {
324	    log.warn('Failed to clear orphan processing claims', { sessionId: session.id, err });
325	  } finally {
326	    if (ownsDb) useDb?.close();
327	  }
328	}
329

/**
 * Host sweep — periodic maintenance of all session DBs.
 *
 * Two-DB architecture:
 *   - Reads processing_ack + container_state from outbound.db
 *   - Writes to inbound.db (host-owned) for status updates + recurrence
 *   - Uses heartbeat file mtime for liveness (never polls DB for it)
 *   - Never writes to outbound.db — preserves single-writer-per-file invariant
 *
 * Stuck / idle detection (replaces the old IDLE_TIMEOUT setTimeout + 10-min
 * heartbeat threshold):
 *
 *   If the container isn't running and there are 'processing' rows left over
 *   (e.g. it crashed mid-turn) → reset them to pending with backoff +
 *   tries++. Existing retry machinery does the rest.
 *
 *   If the container IS running:
 *     1. Absolute ceiling: heartbeat age > max(30 min, current_bash_timeout)
 *        → kill. Covers the "alive but silent for 30 min" case. Extended
 *        only while Bash is declared as running longer, honouring the
 *        user's own timeout directive. Kill then resets processing rows.
 *
 *     2. Message-scoped stuck: for each 'processing' row, tolerance =
 *        max(60s, current_bash_timeout_ms_if_Bash_running). If
 *        (claim_age > tolerance) AND (heartbeat_mtime <= status_changed)
 *        → kill + reset this message + tries++. Semantics: "container
 *        claimed a message and went quiet past tolerance since the claim."
 */
import type Database from 'better-sqlite3';
import fs from 'fs';

import { getActiveSessions } from './db/sessions.js';
import { getAgentGroup } from './db/agent-groups.js';
import {
  countDueMessages,
  deleteOrphanProcessingClaims,
  getContainerState,
  getMessageForRetry,
  getProcessingClaims,
  markMessageFailed,
  retryWithBackoff,
  syncProcessingAcks,
  type ContainerState,
} from './db/session-db.js';
import { log } from './log.js';
import { openInboundDb, openOutboundDb, openOutboundDbRw, inboundDbPath, heartbeatPath } from './session-manager.js';
import { isContainerRunning, killContainer, wakeContainer } from './container-runner.js';
import type { Session } from './types.js';

/**
 * SQLite TIMESTAMP columns store UTC without a timezone marker. Date.parse
 * treats timezoneless ISO strings as local time, so on non-UTC hosts every
 * timestamp looks (TZ offset) hours stale — leading to spurious kill-claim
 * decisions on freshly-claimed messages. Append "Z" when no zone marker is
 * present so Date.parse interprets the string as UTC.
 */
export function parseSqliteUtc(s: string): number {
  return Date.parse(/[zZ]|[+-]\d{2}:?\d{2}$/.test(s) ? s : s + 'Z');
}

const SWEEP_INTERVAL_MS = 60_000;
// Absolute idle ceiling for a running container. If the heartbeat file hasn't
// been touched in this long, the container is either stuck or doing genuinely
// nothing — kill and restart on the next inbound.
export const ABSOLUTE_CEILING_MS = 30 * 60 * 1000;
// Stuck tolerance window applied per 'processing' claim — "did we see any
// signs of life since this message was claimed?"
export const CLAIM_STUCK_MS = 60 * 1000;
const MAX_TRIES = 5;
const BACKOFF_BASE_MS = 5000;

export type StuckDecision =
  | { action: 'ok' }
  | { action: 'kill-ceiling'; heartbeatAgeMs: number; ceilingMs: number }
  | { action: 'kill-claim'; messageId: string; claimAgeMs: number; toleranceMs: number };

/**
 * Pure decision for whether a running container should be killed this sweep
 * tick. Inputs are all deterministic; filesystem + DB reads happen in the
 * caller.
 */
export function decideStuckAction(args: {
  now: number;
  heartbeatMtimeMs: number; // 0 when heartbeat file absent
  containerState: ContainerState | null;
  claims: Array<{ message_id: string; status_changed: string }>;
}): StuckDecision {
  const { now, heartbeatMtimeMs, containerState, claims } = args;
  const declaredBashMs = bashTimeoutMs(containerState);

  // Ceiling check only applies when we have an actual heartbeat timestamp.
  // A freshly-spawned container hasn't had any SDK activity yet so no
  // heartbeat file exists — if we treated that as infinitely stale we'd
  // kill every container within seconds of spawn. Genuinely-dead containers
  // that never wrote a heartbeat are caught by the separate "container
  // process not running" cleanup path, not here. If a fresh container is
  // hanging at the gate (claimed a message but never did anything) the
  // claim-stuck check below handles it.
  if (heartbeatMtimeMs !== 0) {
    const heartbeatAge = now - heartbeatMtimeMs;
    const ceiling = Math.max(ABSOLUTE_CEILING_MS, declaredBashMs ?? 0);
    if (heartbeatAge > ceiling) {
      return { action: 'kill-ceiling', heartbeatAgeMs: heartbeatAge, ceilingMs: ceiling };
    }
  }

  const tolerance = Math.max(CLAIM_STUCK_MS, declaredBashMs ?? 0);
  for (const claim of claims) {
    const claimedAt = parseSqliteUtc(claim.status_changed);
    if (Number.isNaN(claimedAt)) continue;
    const claimAge = now - claimedAt;
    if (claimAge <= tolerance) continue;
    if (heartbeatMtimeMs > claimedAt) continue;
    return { action: 'kill-claim', messageId: claim.message_id, claimAgeMs: claimAge, toleranceMs: tolerance };
  }

  return { action: 'ok' };
}

let running = false;

export function startHostSweep(): void {
  if (running) return;
  running = true;
  sweep();
}

export function stopHostSweep(): void {
  running = false;
}

async function sweep(): Promise<void> {
  if (!running) return;

  try {
    const sessions = getActiveSessions();
    for (const session of sessions) {
      await sweepSession(session);
    }
  } catch (err) {
    log.error('Host sweep error', { err });
  }

  setTimeout(sweep, SWEEP_INTERVAL_MS);
}

async function sweepSession(session: Session): Promise<void> {
  const agentGroup = getAgentGroup(session.agent_group_id);
  if (!agentGroup) return;

  const inPath = inboundDbPath(agentGroup.id, session.id);
  if (!fs.existsSync(inPath)) return;

  let inDb: Database.Database;
  let outDb: Database.Database | null = null;
  try {
    inDb = openInboundDb(agentGroup.id, session.id);
  } catch {
    return;
  }

  try {
    outDb = openOutboundDb(agentGroup.id, session.id);
  } catch {
    // outbound.db might not exist yet (container hasn't started)
  }

  try {
    // 1. Sync processing_ack → messages_in status
    if (outDb) {
      syncProcessingAcks(inDb, outDb);
    }

    // 2. Wake a container if work is due and nothing is running. Ordered
    // before the crashed-container cleanup so a fresh container gets a chance
    // to clean its own orphan processing_ack rows on startup (see
    // container/agent-runner/src/db/connection.ts). Otherwise the reset path
    // would keep bumping process_after into the future, dueCount would stay 0,
    // and the wake would never fire.
    const dueCount = countDueMessages(inDb);
    if (dueCount > 0 && !isContainerRunning(session.id)) {
      log.info('Waking container for due messages', { sessionId: session.id, count: dueCount });
      // wakeContainer never throws — transient spawn failures (OneCLI down,
      // etc.) return false and leave messages pending for the next tick.
      await wakeContainer(session);
    }

    const alive = isContainerRunning(session.id);

    // 3. Running-container SLA: absolute ceiling + per-claim stuck rules.
    if (alive && outDb) {
      enforceRunningContainerSla(inDb, outDb, session, agentGroup.id);
    }

    // 4. Crashed-container cleanup: processing rows left behind get retried.
    // Only fires when wake in step 2 didn't pick up the work (no due messages,
    // or wake failed). resetStuckProcessingRows itself is idempotent — it
    // skips messages already scheduled for a future retry.
    if (!alive && outDb) {
      resetStuckProcessingRows(inDb, outDb, session, 'container not running');
    }

    // 5. Recurrence fanout for completed recurring tasks.
    // MODULE-HOOK:scheduling-recurrence:start
    const { handleRecurrence } = await import('./modules/scheduling/recurrence.js');
    await handleRecurrence(inDb, session);
    // MODULE-HOOK:scheduling-recurrence:end
  } finally {
    inDb.close();
    outDb?.close();
  }
}

function heartbeatMtimeMs(agentGroupId: string, sessionId: string): number {
  const hbPath = heartbeatPath(agentGroupId, sessionId);
  try {
    return fs.statSync(hbPath).mtimeMs;
  } catch {
    return 0;
  }
}

function bashTimeoutMs(state: ContainerState | null): number | null {
  if (!state || state.current_tool !== 'Bash') return null;
  return typeof state.tool_declared_timeout_ms === 'number' ? state.tool_declared_timeout_ms : null;
}

function enforceRunningContainerSla(
  inDb: Database.Database,
  outDb: Database.Database,
  session: Session,
  agentGroupId: string,
): void {
  const decision = decideStuckAction({
    now: Date.now(),
    heartbeatMtimeMs: heartbeatMtimeMs(agentGroupId, session.id),
    containerState: getContainerState(outDb),
    claims: getProcessingClaims(outDb),
  });

  if (decision.action === 'ok') return;

  if (decision.action === 'kill-ceiling') {
    log.warn('Killing container past absolute ceiling', {
      sessionId: session.id,
      heartbeatAgeMs: decision.heartbeatAgeMs,
      ceilingMs: decision.ceilingMs,
    });
    killContainer(session.id, 'absolute-ceiling');
    resetStuckProcessingRows(inDb, outDb, session, 'absolute-ceiling');
    return;
  }

  log.warn('Killing container — message claimed then silent', {
    sessionId: session.id,
    messageId: decision.messageId,
    claimAgeMs: decision.claimAgeMs,
    toleranceMs: decision.toleranceMs,
  });
  killContainer(session.id, 'claim-stuck');
  resetStuckProcessingRows(inDb, outDb, session, 'claim-stuck');
}

export function _resetStuckProcessingRowsForTesting(
  inDb: Database.Database,
  outDb: Database.Database,
  session: Session,
  reason: string,
): void {
  resetStuckProcessingRows(inDb, outDb, session, reason, outDb);
}

function resetStuckProcessingRows(
  inDb: Database.Database,
  outDb: Database.Database,
  session: Session,
  reason: string,
  writableOutDb?: Database.Database,
): void {
  const claims = getProcessingClaims(outDb);
  const now = Date.now();
  for (const { message_id } of claims) {
    const msg = getMessageForRetry(inDb, message_id, 'pending');
    if (!msg) continue;

    // Already rescheduled for a future retry — don't bump tries again. The
    // wake path (sweep step 2) will fire when process_after elapses and a
    // fresh container will clean the orphan claim on startup.
    if (msg.processAfter && parseSqliteUtc(msg.processAfter) > now) continue;

    if (msg.tries >= MAX_TRIES) {
      markMessageFailed(inDb, msg.id);
      log.warn('Message marked as failed after max retries', {
        messageId: msg.id,
        sessionId: session.id,
        reason,
      });
    } else {
      const backoffMs = BACKOFF_BASE_MS * Math.pow(2, msg.tries);
      const backoffSec = Math.floor(backoffMs / 1000);
      retryWithBackoff(inDb, msg.id, backoffSec);
      log.info('Reset stale message with backoff', {
        messageId: msg.id,
        tries: msg.tries,
        backoffMs,
        reason,
      });
    }
  }

  // Drop the orphan 'processing' rows. Without this, the next sweep tick
  // would re-read them, see the old status_changed timestamp, conclude the
  // freshly respawned container is stuck, and SIGKILL it before its
  // agent-runner has a chance to run clearStaleProcessingAcks() on startup.
  const ownsDb = !writableOutDb;
  let useDb: Database.Database | null = writableOutDb ?? null;
  try {
    if (!useDb) useDb = openOutboundDbRw(session.agent_group_id, session.id);
    const cleared = deleteOrphanProcessingClaims(useDb);
    if (cleared > 0) {
      log.info('Cleared orphan processing claims', { sessionId: session.id, cleared, reason });
    }
  } catch (err) {
    log.warn('Failed to clear orphan processing claims', { sessionId: session.id, err });
  } finally {
    if (ownsDb) useDb?.close();
  }
}


OK now I understand the heartbeat mechanism. Let me look at the container-runner to understand where the heartbeat is written.

The observation says: "Recurring `Killing container past absolute ceiling` (heartbeatAgeMs ≈ 30 min) for the main DM session several times a day — containers hang past the heartbeat ceiling; cause not yet investigated."

The `heartbeatAgeMs ≈ 30 min` and `ceilingMs = 30 min` means the heartbeat is EXACTLY at the ceiling when it gets killed. This strongly suggests the heartbeat is NOT being updated during long-running agent turns.
Now I need to trace where the heartbeat file gets touched during execution. The ceiling is set to 30 minutes, and the heartbeat is checked by looking at the file's modification time — if it hasn't been updated within that window, the container gets terminated. So I need to find where in the container-runner and agent-runner the heartbeat is actually being refreshed.


grep -rn "heartbeat\|\.heartbeat\|touchHeartbeat\|heartbeatPath\|touch.*heartbeat" /workspace/extra/projects/nanoclaw/container/agent-runner/src --include="*.ts" 2>/dev/null | head -40

Find heartbeat writes in agent-runner


/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:4:import { getInboundDb, touchHeartbeat, clearStaleProcessingAcks } from './db/connection.js';
/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:126:    // Periodic heartbeat so we know the loop is alive
/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:128:      log(`Poll heartbeat (${pollCount} iterations, ${messages.length} pending)`);
/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:334:  // Stream liveness is decided host-side via the heartbeat file + processing
/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:425:            // Stop touching the heartbeat so host-sweep stale detection fires
/workspace/extra/projects/nanoclaw/container/agent-runner/src/poll-loop.ts:448:      touchHeartbeat();
/workspace/extra/projects/nanoclaw/container/agent-runner/src/scheduling/task-script.ts:5:import { touchHeartbeat } from '../db/connection.js';
/workspace/extra/projects/nanoclaw/container/agent-runner/src/scheduling/task-script.ts:104:    touchHeartbeat();
/workspace/extra/projects/nanoclaw/container/agent-runner/src/scheduling/task-script.ts:106:    touchHeartbeat();
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/connection.ts:25:const DEFAULT_HEARTBEAT_PATH = '/workspace/.heartbeat';
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/connection.ts:29:let _heartbeatPath: string = DEFAULT_HEARTBEAT_PATH;
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/connection.ts:152: * Touch the heartbeat file — replaces the old touchProcessing() DB writes.
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/connection.ts:156:export function touchHeartbeat(): void {
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/connection.ts:157:  const p = _heartbeatPath;
/workspace/extra/projects/nanoclaw/container/agent-runner/src/db/index.ts:7:  touchHeartbeat,
/workspace/extra/projects/nanoclaw/container/agent-runner/src/index.ts:14: *     .heartbeat        ← container touches for liveness detection


110	  }
111
112	  // Clear leftover 'processing' acks from a previous crashed container.
113	  // This lets the new container re-process those messages.
114	  clearStaleProcessingAcks();
115
116	  let pollCount = 0;
117	  let isFirstPoll = true;
118	  while (true) {
119	    if (config.signal?.aborted) return;
120
121	    // Skip system messages — they're responses for MCP tools (e.g., ask_user_question)
122	    const messages = getPendingMessages(isFirstPoll).filter((m) => m.kind !== 'system');
123	    isFirstPoll = false;
124	    pollCount++;
125
126	    // Periodic heartbeat so we know the loop is alive
127	    if (pollCount % 30 === 0) {
128	      log(`Poll heartbeat (${pollCount} iterations, ${messages.length} pending)`);
129	    }
130
131	    if (messages.length === 0) {
132	      await sleep(POLL_INTERVAL_MS);
133	      continue;
134	    }
135
136	    // Accumulate gate: if the batch contains only trigger=0 rows
137	    // (context-only, router-stored under ignored_message_policy='accumulate'),
138	    // don't wake the agent. Leave them `pending` — they'll ride along the
139	    // next time a real trigger=1 message lands via this same getPendingMessages
140	    // query. Without this gate, a warm container keeps processing
141	    // (and potentially responding to) every accumulate-only batch, defeating
142	    // the "store as context, don't engage" contract. Host-side countDueMessages
143	    // gates the same way for wake-from-cold (see src/db/session-db.ts).
144	    if (!messages.some((m) => m.trigger === 1)) {
145	      await sleep(POLL_INTERVAL_MS);
146	      continue;
147	    }
148
149	    const ids = messages.map((m) => m.id);
150	    markProcessing(ids);
151
152	    const routing = extractRouting(messages);
153
154	    // Command handling: the host router gates filtered and unauthorized
155	    // admin commands before they reach the container. The only command
156	    // the runner handles directly is /clear (session reset).
157	    const normalMessages: MessageInRow[] = [];
158	    const commandIds: string[] = [];
159
160	    for (const msg of messages) {
161	      if ((msg.kind === 'chat' || msg.kind === 'chat-sdk') && isClearCommand(msg)) {
162	        log('Clearing session (resetting continuation)');
163	        continuation = undefined;
164	        clearContinuation(config.providerName);
165	        writeMessageOut({
166	          id: generateId(),
167	          kind: 'chat',
168	          platform_id: routing.platformId,
169	          channel_type: routing.channelType,
170	          thread_id: routing.threadId,
171	          content: JSON.stringify({ text: 'Session cleared.' }),
172	        });
173	        commandIds.push(msg.id);
174	        continue;
175	      }
176	      normalMessages.push(msg);
177	    }
178
179	    if (commandIds.length > 0) {
180	      markCompleted(commandIds);
181	    }
182
183	    if (normalMessages.length === 0) {
184	      const remainingIds = ids.filter((id) => !commandIds.includes(id));
185	      if (remainingIds.length > 0) markCompleted(remainingIds);
186	      log(`All ${messages.length} message(s) were commands, skipping query`);
187	      continue;
188	    }
189

  }

  // Clear leftover 'processing' acks from a previous crashed container.
  // This lets the new container re-process those messages.
  clearStaleProcessingAcks();

  let pollCount = 0;
  let isFirstPoll = true;
  while (true) {
    if (config.signal?.aborted) return;

    // Skip system messages — they're responses for MCP tools (e.g., ask_user_question)
    const messages = getPendingMessages(isFirstPoll).filter((m) => m.kind !== 'system');
    isFirstPoll = false;
    pollCount++;

    // Periodic heartbeat so we know the loop is alive
    if (pollCount % 30 === 0) {
      log(`Poll heartbeat (${pollCount} iterations, ${messages.length} pending)`);
    }

    if (messages.length === 0) {
      await sleep(POLL_INTERVAL_MS);
      continue;
    }

    // Accumulate gate: if the batch contains only trigger=0 rows
    // (context-only, router-stored under ignored_message_policy='accumulate'),
    // don't wake the agent. Leave them `pending` — they'll ride along the
    // next time a real trigger=1 message lands via this same getPendingMessages
    // query. Without this gate, a warm container keeps processing
    // (and potentially responding to) every accumulate-only batch, defeating
    // the "store as context, don't engage" contract. Host-side countDueMessages
    // gates the same way for wake-from-cold (see src/db/session-db.ts).
    if (!messages.some((m) => m.trigger === 1)) {
      await sleep(POLL_INTERVAL_MS);
      continue;
    }

    const ids = messages.map((m) => m.id);
    markProcessing(ids);

    const routing = extractRouting(messages);

    // Command handling: the host router gates filtered and unauthorized
    // admin commands before they reach the container. The only command
    // the runner handles directly is /clear (session reset).
    const normalMessages: MessageInRow[] = [];
    const commandIds: string[] = [];

    for (const msg of messages) {
      if ((msg.kind === 'chat' || msg.kind === 'chat-sdk') && isClearCommand(msg)) {
        log('Clearing session (resetting continuation)');
        continuation = undefined;
        clearContinuation(config.providerName);
        writeMessageOut({
          id: generateId(),
          kind: 'chat',
          platform_id: routing.platformId,
          channel_type: routing.channelType,
          thread_id: routing.threadId,
          content: JSON.stringify({ text: 'Session cleared.' }),
        });
        commandIds.push(msg.id);
        continue;
      }
      normalMessages.push(msg);
    }

    if (commandIds.length > 0) {
      markCompleted(commandIds);
    }

    if (normalMessages.length === 0) {
      const remainingIds = ids.filter((id) => !commandIds.includes(id));
      if (remainingIds.length > 0) markCompleted(remainingIds);
      log(`All ${messages.length} message(s) were commands, skipping query`);
      continue;
    }