Read the congressional-trading repo at https://github.com/jamesjlopez/congressional-trading via the GitHub API or by reading local files at ~/projects/congressional-trading if accessible.

I need to understand:
1. What the scoring/signal system does — what makes a trade "high conviction"? What are the scoring criteria or thresholds?
2. What does a signal notification look like — what data does it include?
3. Is there a shadow measurement mode? What does it track/log?
4. Is there any historical trade data stored, or does it only process new disclosures?
5. What are the exit criteria or thresholds that would graduate a signal from shadow to real?
6. Are there any example signals or test fixtures in the codebase?

Also look at the Python source files for the core scoring logic. I want to understand the algorithm well enough to assess whether we can backtest it against historical congressional trade data.

Return a structured summary of: the scoring algorithm, what triggers a notification, what data is in shadow mode, and any obvious gaps or open decisions visible in the code (TODOs, hardcoded values, etc.).


I'll help you understand the congressional-trading repository. Let me start by checking if the local project exists and then examining its structure.


Bash

ls -la ~/projects/ 2>/dev/null | head -20


(Bash completed with no output)


Let me try to fetch the repository via GitHub API using the onecli-gateway skill.


Launching skill: onecli-gateway


Base directory for this skill: /home/node/.claude/skills/onecli-gateway

# OneCLI Gateway

Your outbound HTTPS traffic is transparently proxied through the OneCLI
gateway, which injects stored credentials at the proxy boundary. You never
see or handle credential values directly.

## How to Access External Services

You have direct HTTP access to external APIs. OAuth apps (Gmail, GitHub,
Google Calendar, Google Drive, etc.) and API key services are all available
through the gateway. Just make the request directly; the gateway injects
credentials if the app is connected. If not, it returns an error with a
connect URL you can present to the user.

## Making Requests

Call the real API URL. The gateway intercepts the request and injects
credentials automatically.

```bash
curl -s "https://gmail.googleapis.com/gmail/v1/users/me/messages?maxResults=5"
curl -s "https://api.github.com/user/repos?per_page=10"
curl -s "https://api.stripe.com/v1/charges?limit=5"
```

Standard HTTP clients (curl, fetch, requests, axios, Go net/http, git) all
honor the `HTTPS_PROXY` environment variable automatically. You do not need
to set any auth headers.

## Credential Stubs for MCP Servers

Some MCP servers need local credential files to start. Stubs for connected
apps are pre-written automatically. Files containing `"onecli-managed"`
values are managed by OneCLI — do NOT modify or delete them.

If an MCP server won't start due to missing credentials, create stubs
**before** starting it. Use `"onecli-managed"` as the placeholder for all
secret values, with file permissions `0600`. See the guide at:
https://www.onecli.sh/docs/guides/credential-stubs/general-app

## When a Request Fails

If you get a 401, 403, or a gateway error (e.g., `app_not_connected`):

**Step 1 — Show the user a connect link.** Use the `connect_url` from the
error response:

> To connect [service], open this link:
> [connect_url from the error response]

If there is no `connect_url` in the error, tell the user to open the
OneCLI dashboard and connect the service there.

**Step 2 — Retry after the user connects.** Let the user know you will
retry once they have connected. When they confirm, retry the original
request. If the retry still fails, ask if they need help with the setup.

## Rules

- **Never** say "I don't have access to X" without first making the HTTP
  request through the proxy.
- **Never** use browser extensions, gcloud, or manual auth flows. The
  gateway handles credentials for you.
- **Never** ask the user for API keys or tokens directly. Direct them to
  connect the service in the OneCLI dashboard.
- **Never** suggest the user open Gmail/Calendar/GitHub in their browser
  when they ask you to read or interact with those services. You have API
  access. Use it.
- If the gateway returns a policy error (403 with a JSON body), respect
  the block. Do not retry or circumvent it.

ARGUMENTS: GET https://api.github.com/repos/jamesjlopez/congressional-trading


I'll fetch the repository information and explore its structure via the GitHub API.


curl -s "https://api.github.com/repos/jamesjlopez/congressional-trading" | head -100


{
  "id": 1219398569,
  "node_id": "R_kgDOSK6LqQ",
  "name": "congressional-trading",
  "full_name": "jamesjlopez/congressional-trading",
  "private": false,
  "owner": {
    "login": "jamesjlopez",
    "id": 5341989,
    "node_id": "MDQ6VXNlcjUzNDE5ODk=",
    "avatar_url": "https://avatars.githubusercontent.com/u/5341989?v=4",
    "gravatar_id": "",
    "url": "https://api.github.com/users/jamesjlopez",
    "html_url": "https://github.com/jamesjlopez",
    "followers_url": "https://api.github.com/users/jamesjlopez/followers",
    "following_url": "https://api.github.com/users/jamesjlopez/following{/other_user}",
    "gists_url": "https://api.github.com/users/jamesjlopez/gists{/gist_id}",
    "starred_url": "https://api.github.com/users/jamesjlopez/starred{/owner}{/repo}",
    "subscriptions_url": "https://api.github.com/users/jamesjlopez/subscriptions",
    "organizations_url": "https://api.github.com/users/jamesjlopez/orgs",
    "repos_url": "https://api.github.com/users/jamesjlopez/repos",
    "events_url": "https://api.github.com/users/jamesjlopez/events{/privacy}",
    "received_events_url": "https://api.github.com/users/jamesjlopez/received_events",
    "type": "User",
    "user_view_type": "public",
    "site_admin": false
  },
  "html_url": "https://github.com/jamesjlopez/congressional-trading",
  "description": "Congressional members frequently trade stocks while having access to non-public policy information and legislation in progress. The STOCK Act (2012) mandates public disclosure of trades over $1,000 within 45 days. This creates a legally available, high-quality signal stream that most retail investors ignore or consume passively.",
  "fork": false,
  "url": "https://api.github.com/repos/jamesjlopez/congressional-trading",
  "forks_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/forks",
  "keys_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/keys{/key_id}",
  "collaborators_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/collaborators{/collaborator}",
  "teams_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/teams",
  "hooks_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/hooks",
  "issue_events_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/issues/events{/number}",
  "events_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/events",
  "assignees_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/assignees{/user}",
  "branches_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/branches{/branch}",
  "tags_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/tags",
  "blobs_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs{/sha}",
  "git_tags_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/tags{/sha}",
  "git_refs_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/refs{/sha}",
  "trees_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees{/sha}",
  "statuses_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/statuses/{sha}",
  "languages_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/languages",
  "stargazers_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/stargazers",
  "contributors_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/contributors",
  "subscribers_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/subscribers",
  "subscription_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/subscription",
  "commits_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/commits{/sha}",
  "git_commits_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/commits{/sha}",
  "comments_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/comments{/number}",
  "issue_comment_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/issues/comments{/number}",
  "contents_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/contents/{+path}",
  "compare_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/compare/{base}...{head}",
  "merges_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/merges",
  "archive_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/{archive_format}{/ref}",
  "downloads_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/downloads",
  "issues_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/issues{/number}",
  "pulls_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/pulls{/number}",
  "milestones_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/milestones{/number}",
  "notifications_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/notifications{?since,all,participating}",
  "labels_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/labels{/name}",
  "releases_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/releases{/id}",
  "deployments_url": "https://api.github.com/repos/jamesjlopez/congressional-trading/deployments",
  "created_at": "2026-04-23T20:50:02Z",
  "updated_at": "2026-06-17T22:38:14Z",
  "pushed_at": "2026-06-17T22:36:42Z",
  "git_url": "git://github.com/jamesjlopez/congressional-trading.git",
  "ssh_url": "git@github.com:jamesjlopez/congressional-trading.git",
  "clone_url": "https://github.com/jamesjlopez/congressional-trading.git",
  "svn_url": "https://github.com/jamesjlopez/congressional-trading",
  "homepage": null,
  "size": 1130,
  "stargazers_count": 0,
  "watchers_count": 0,
  "language": "Python",
  "has_issues": true,
  "has_projects": true,
  "has_downloads": true,
  "has_wiki": false,
  "has_pages": false,
  "has_discussions": false,
  "forks_count": 0,
  "mirror_url": null,
  "archived": false,
  "disabled": false,
  "open_issues_count": 0,
  "license": null,
  "allow_forking": true,
  "is_template": false,
  "web_commit_signoff_required": false,
  "has_pull_requests": true,
  "pull_request_creation_policy": "all",
  "topics": [

  ],
  "visibility": "public",


Good! Now let me get the file structure of the repository.


curl -s "https://api.github.com/repos/jamesjlopez/congressional-trading/contents/" | jq -r '.[] | "\(.type): \(.name)"'


Error: Exit code 127
/bin/bash: line 1: jq: command not found


curl -s "https://api.github.com/repos/jamesjlopez/congressional-trading/contents/" | grep -o '"name":"[^"]*"' | head -20


Let me try a different approach - clone the repo and explore it locally.


cd /tmp && git clone https://github.com/jamesjlopez/congressional-trading.git 2>&1 | head -20


Cloning into 'congressional-trading'...
fatal: unable to access 'https://github.com/jamesjlopez/congressional-trading.git/': server certificate verification failed. CAfile: none CRLfile: none


Let me try using curl to get the raw file contents directly via the GitHub API.


curl -s "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/main?recursive=1" | head -200


{
  "sha": "2ef587b5151b1d3e84088b2c8dcc3dc90737db31",
  "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/2ef587b5151b1d3e84088b2c8dcc3dc90737db31",
  "tree": [
    {
      "path": ".env.example",
      "mode": "100644",
      "type": "blob",
      "sha": "98f5f8958ab1db362b8ff3b8a012ceecee87bdca",
      "size": 372,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/98f5f8958ab1db362b8ff3b8a012ceecee87bdca"
    },
    {
      "path": ".gitignore",
      "mode": "100644",
      "type": "blob",
      "sha": "5d280d1b099f5d60d919737886f75537d5c78de2",
      "size": 421,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/5d280d1b099f5d60d919737886f75537d5c78de2"
    },
    {
      "path": "CLAUDE.md",
      "mode": "100644",
      "type": "blob",
      "sha": "7e95a6796f7be634c247567d74af5a806ad66dc2",
      "size": 5343,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/7e95a6796f7be634c247567d74af5a806ad66dc2"
    },
    {
      "path": "README.md",
      "mode": "100644",
      "type": "blob",
      "sha": "cd65dc08ccfdfc79fef376c09e32818892f0521d",
      "size": 10070,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/cd65dc08ccfdfc79fef376c09e32818892f0521d"
    },
    {
      "path": "analysis",
      "mode": "040000",
      "type": "tree",
      "sha": "8142d7b91c760712b647218a214d08608f7a2e86",
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/8142d7b91c760712b647218a214d08608f7a2e86"
    },
    {
      "path": "analysis/__init__.py",
      "mode": "100644",
      "type": "blob",
      "sha": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391",
      "size": 0,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"
    },
    {
      "path": "analysis/llm_client.py",
      "mode": "100644",
      "type": "blob",
      "sha": "d7141d05e5203fd3a3dce7b9d5122a82b0d63afd",
      "size": 3151,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/d7141d05e5203fd3a3dce7b9d5122a82b0d63afd"
    },
    {
      "path": "analysis/notification_policy.py",
      "mode": "100644",
      "type": "blob",
      "sha": "f78abeba40e368b309a8fdb5ee2a298a4a83d75f",
      "size": 5772,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/f78abeba40e368b309a8fdb5ee2a298a4a83d75f"
    },
    {
      "path": "analysis/predictor.py",
      "mode": "100644",
      "type": "blob",
      "sha": "535415da3b8f7453b8fd334034abcd36e2b9b35f",
      "size": 28845,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/535415da3b8f7453b8fd334034abcd36e2b9b35f"
    },
    {
      "path": "analysis/researcher.py",
      "mode": "100644",
      "type": "blob",
      "sha": "25a1fdf79d1d8b8361ed1f200d4459dab145e09b",
      "size": 21502,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/25a1fdf79d1d8b8361ed1f200d4459dab145e09b"
    },
    {
      "path": "analysis/scorer.py",
      "mode": "100644",
      "type": "blob",
      "sha": "59504d64360da58f6a4baaab0bbe632c14e98a6d",
      "size": 16144,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/59504d64360da58f6a4baaab0bbe632c14e98a6d"
    },
    {
      "path": "analysis/timeliness.py",
      "mode": "100644",
      "type": "blob",
      "sha": "8051022f6bada709b06ff3b6d19cd483017f640f",
      "size": 1108,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/8051022f6bada709b06ff3b6d19cd483017f640f"
    },
    {
      "path": "app-icon.png",
      "mode": "100644",
      "type": "blob",
      "sha": "5985f085f989548a5de6e0e5cdd473e6d2bab941",
      "size": 601002,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/5985f085f989548a5de6e0e5cdd473e6d2bab941"
    },
    {
      "path": "backlog.md",
      "mode": "100644",
      "type": "blob",
      "sha": "0622250129abf110c320d551ba193f31a50d2329",
      "size": 5069,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/0622250129abf110c320d551ba193f31a50d2329"
    },
    {
      "path": "config.py",
      "mode": "100644",
      "type": "blob",
      "sha": "358d2df042155fd894ef5dc68ae0a009eee73ab5",
      "size": 3366,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/358d2df042155fd894ef5dc68ae0a009eee73ab5"
    },
    {
      "path": "data",
      "mode": "040000",
      "type": "tree",
      "sha": "4b06c6655c70185b50cc5e6614d6907281e19547",
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/4b06c6655c70185b50cc5e6614d6907281e19547"
    },
    {
      "path": "data/.gitkeep",
      "mode": "100644",
      "type": "blob",
      "sha": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391",
      "size": 0,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"
    },
    {
      "path": "data/committee_membership.yaml",
      "mode": "100644",
      "type": "blob",
      "sha": "2b5db77ba3e21a4517bb1542b36d7cb6c9404969",
      "size": 291212,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/2b5db77ba3e21a4517bb1542b36d7cb6c9404969"
    },
    {
      "path": "data/committees_current.yaml",
      "mode": "100644",
      "type": "blob",
      "sha": "b54a0ae03c347bf24df80581f5486b1dc3ac835f",
      "size": 63366,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/b54a0ae03c347bf24df80581f5486b1dc3ac835f"
    },
    {
      "path": "db",
      "mode": "040000",
      "type": "tree",
      "sha": "2c1f5ff695d51b75bf87e96926ead9e93654e91e",
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/2c1f5ff695d51b75bf87e96926ead9e93654e91e"
    },
    {
      "path": "db/migrate.py",
      "mode": "100644",
      "type": "blob",
      "sha": "95c656a912be88f25abf88a0c69b6587a48351f9",
      "size": 4996,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/95c656a912be88f25abf88a0c69b6587a48351f9"
    },
    {
      "path": "db/schema.sql",
      "mode": "100644",
      "type": "blob",
      "sha": "7ce8118a7b7161c4fbbb23230206b0d541f909f8",
      "size": 5906,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/7ce8118a7b7161c4fbbb23230206b0d541f909f8"
    },
    {
      "path": "ingest",
      "mode": "040000",
      "type": "tree",
      "sha": "89ff2076d1a5c2ccc2ad0ee24cfb3629cedb2219",
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/trees/89ff2076d1a5c2ccc2ad0ee24cfb3629cedb2219"
    },
    {
      "path": "ingest/__init__.py",
      "mode": "100644",
      "type": "blob",
      "sha": "e69de29bb2d1d6434b8b29ae775ad8c2e48c5391",
      "size": 0,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/e69de29bb2d1d6434b8b29ae775ad8c2e48c5391"
    },
    {
      "path": "ingest/capitol_trades.py",
      "mode": "100644",
      "type": "blob",
      "sha": "dd00b7abd007161bc44e48dd31a42b298aaff11a",
      "size": 9640,
      "url": "https://api.github.com/repos/jamesjlopez/congressional-trading/git/blobs/dd00b7abd007161bc44e48dd31a42b298aaff11a"
    },


Great! Now let me fetch the key Python files to understand the scoring algorithm and signal system. Let me get the raw content of the most important files.


"""
Signal scoring engine.

Groups trades by ticker within a lookback window and produces a score
for each ticker reflecting the aggregate congressional activity.

Score components (max ~13):
  committee_score    0–3   member's committee overlaps with company sector (Day 5)
  track_record_score 0–2   historical win rate (defaults to 1.0 until calculable)
  cluster_score      0–3   unique member buyers (1=0pts, 2=1pt, 3=2pts, 4+=3pts)
  options_bonus      0–2   any option trade on this ticker (+2)
  size_score         0–5   highest dollar tier among buy trades

Notification threshold (SCORE_NOTIFY_THRESHOLD in config.py): default 7
"""

from __future__ import annotations
import sqlite3
import sys
import os
from collections import defaultdict
from datetime import datetime, timedelta

sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from config import DB_PATH, LOOKBACK_DAYS, SCORE_NOTIFY_THRESHOLD

# Committee → sector mapping used for committee_score.
# Updated when committees.py runs (Day 5); used here for scoring.
# Key: normalized committee name fragment, Value: list of matching sectors
_COMMITTEE_SECTOR_MAP = {
    "armed services":       ["industrials", "information-technology"],
    "defense":              ["industrials", "information-technology"],
    "intelligence":         ["information-technology", "industrials"],
    "energy":               ["energy", "utilities", "materials"],
    "natural resources":    ["energy", "materials"],
    "science":              ["information-technology"],
    "technology":           ["information-technology", "communication-services"],
    "health":               ["health-care"],
    "financial services":   ["financials"],
    "banking":              ["financials"],
    "finance":              ["financials"],
    "agriculture":          ["consumer-staples", "materials"],
    "transportation":       ["industrials"],
    "commerce":             ["consumer-discretionary", "communication-services"],
    "judiciary":            [],
    "foreign affairs":      [],
    "ways and means":       ["financials"],
}

def _committee_score(committees_json: str | None, sector: str) -> float:
    """
    Return 0–3 based on overlap between member's committees and company sector.
    0 = no data or no overlap
    1 = indirect overlap (committee has broad relevance)
    2 = moderate overlap
    3 = direct jurisdiction
    """
    if not committees_json or not sector:
        return 0.0
    import json
    try:
        committees = json.loads(committees_json)
    except (json.JSONDecodeError, TypeError):
        return 0.0

    sector = sector.lower()
    best = 0.0
    for committee in committees:
        name = committee.lower() if isinstance(committee, str) else (committee.get("name", "") or "").lower()
        for key, sectors in _COMMITTEE_SECTOR_MAP.items():
            if key in name:
                if sector in sectors:
                    best = max(best, 3.0)
                elif any(s in sector for s in ["tech", "info", "comm"]) and key in ("science", "technology", "commerce"):
                    best = max(best, 2.0)
                else:
                    best = max(best, 1.0)
    return best

def _cluster_score(unique_buyers: int) -> float:
    if unique_buyers >= 4:
        return 3.0
    if unique_buyers == 3:
        return 2.0
    if unique_buyers == 2:
        return 1.0
    return 0.0

def _size_score(max_tier: int) -> float:
    """tier is already 1–6 from ingest; cap contribution at 5."""
    return min(5.0, float(max(0, max_tier)))

def _sell_pressure(unique_sellers: int, unique_buyers: int) -> float:
    """
    Penalty when congressional members are selling a ticker they or others are buying.
    The ratio of sellers to buyers determines severity:

    sellers == 0:                0.0  — clean signal
    ratio < 0.40 (e.g. 1 of 3): -0.5 — minor noise, buyers dominate
    ratio < 0.67 (e.g. 1 of 2): -1.5 — worth noting
    ratio < 1.0  (e.g. 2 of 3): -2.5 — mixed conviction
    ratio >= 1.0 (sellers ≥ buyers): -4.0 — sell pressure outweighs buys
    """
    if unique_sellers == 0:
        return 0.0
    ratio = unique_sellers / max(unique_buyers, 1)
    if ratio >= 1.0:
        return -4.0
    if ratio >= 0.67:
        return -2.5
    if ratio >= 0.40:
        return -1.5
    return -0.5

def _small_cap_bonus(market_cap: float | None) -> float:
    """
    Score bonus/penalty based on market cap.

    The thesis: congressional informed buying in a smaller, less-covered company
    has higher percentage-move potential than the same signal in a mega-cap where
    hundreds of analysts are already pricing in every catalyst. Tiers:

    Mega  (>$100B):  -2.5  — saturated coverage; edge is minimal
    Large ($10-100B): -0.5  — well-covered; slight penalty
    Mid   ($2-10B):  +2.0  — meaningful analyst coverage gap
    Small ($500M-$2B): +4.5 — thin coverage, catalysts move the stock
    Micro (<$500M):  +6.0  — very thin float; single catalyst → 30-50% moves
    Unknown:          0.0  — neutral until price_fetcher populates market_cap
    """
    if market_cap is None:
        return 0.0
    if market_cap > 100e9:
        return -2.5
    if market_cap > 10e9:
        return -0.5
    if market_cap > 2e9:
        return 2.0
    if market_cap > 500e6:
        return 4.5
    return 6.0

def _breadth_mult(hist_buyers: int) -> float:
    """
    Discount factor applied to cluster_score based on how many unique members
    have historically bought a ticker. Mega-caps routinely bought by 15+ members
    are portfolio staples, not informed signals — their cluster score overstates edge.
    """
    if hist_buyers >= 20:
        return 0.40   # basically an index holding (MSFT, GOOGL, NVDA tier)
    if hist_buyers >= 12:
        return 0.65   # widely held (HD, BRK/B, GS tier)
    if hist_buyers >= 6:
        return 0.85   # moderately popular
    return 1.0        # rare buy — full cluster score

def score_tickers(lookback_days: int = LOOKBACK_DAYS) -> list[dict]:
    """
    Score all tickers with at least one BUY trade published in the last
    lookback_days. Returns list of signal dicts sorted by total_score desc.
    Also upserts results into the signals table.
    """
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row

    cutoff = (datetime.now() - timedelta(days=lookback_days)).strftime("%Y-%m-%d")

    # Cached market caps from previous price_fetcher run (neutral default if missing)
    cap_rows = conn.execute("SELECT ticker, market_cap FROM signals WHERE market_cap IS NOT NULL").fetchall()
    market_caps: dict[str, float] = {r["ticker"]: r["market_cap"] for r in cap_rows}

    # LLM-derived scores from research_cache — default 0 until research has run
    rc_rows = conn.execute(
        "SELECT ticker, COALESCE(strategic_score,0) AS ss, COALESCE(disruption_score,0) AS ds FROM research_cache"
    ).fetchall()
    llm_scores: dict[str, tuple[float, float]] = {r["ticker"]: (r["ss"], r["ds"]) for r in rc_rows}

    # Historical buyer breadth — full DB, not windowed — used for breadth discount
    breadth_rows = conn.execute(
        """
        SELECT issuer_ticker, COUNT(DISTINCT member_id) AS cnt
        FROM trades
        WHERE tx_type = 'buy' AND issuer_ticker != ''
        GROUP BY issuer_ticker
        """
    ).fetchall()
    historical_buyers: dict[str, int] = {r["issuer_ticker"]: r["cnt"] for r in breadth_rows}

    rows = conn.execute(
        """
        SELECT
            t.tx_id,
            t.issuer_ticker,
            t.issuer_name,
            t.issuer_sector,
            t.member_id,
            t.tx_type,
            t.amount_tier,
            t.is_option,
            t.pub_date,
            t.tx_date,
            t.value_estimate,
            m.party,
            m.chamber,
            m.first_name || ' ' || m.last_name AS member_name,
            m.committees,
            COALESCE(m.win_rate, -1) AS win_rate
        FROM trades t
        JOIN members m ON m.bioguide_id = t.member_id
        WHERE t.pub_date >= ?
          AND t.issuer_ticker != ''
        ORDER BY t.issuer_ticker, t.pub_date DESC
        """,
        (cutoff,),
    ).fetchall()

    # Group by ticker
    by_ticker: dict[str, list] = defaultdict(list)
    for row in rows:
        by_ticker[row["issuer_ticker"]].append(dict(row))

    signals = []

    for ticker, ticker_rows in by_ticker.items():
        buy_rows = [r for r in ticker_rows if r["tx_type"] == "buy"]
        sell_rows = [r for r in ticker_rows if r["tx_type"] == "sell"]
        option_rows = [r for r in ticker_rows if r["is_option"]]
        sell_ids = set(r["member_id"] for r in sell_rows)

        if not buy_rows:
            continue  # only score tickers with at least one buy

        # --- Cluster score: unique member buyers, discounted for mega-caps ---
        buyer_ids = set(r["member_id"] for r in buy_rows)
        unique_buyers = len(buyer_ids)
        hist = historical_buyers.get(ticker, 0)
        cluster = round(_cluster_score(unique_buyers) * _breadth_mult(hist), 2)

        # --- Size score: max tier among buys ---
        max_tier = max((r["amount_tier"] or 0) for r in buy_rows)
        size = _size_score(max_tier)

        # --- Options bonus ---
        opt_bonus = 2.0 if option_rows else 0.0

        # --- Track record score (default 1 until win_rate data available) ---
        win_rates = [r["win_rate"] for r in buy_rows if r["win_rate"] >= 0]
        if win_rates:
            avg_wr = sum(win_rates) / len(win_rates)
            track = 2.0 if avg_wr > 0.6 else (1.0 if avg_wr >= 0.5 else 0.5)
        else:
            track = 1.0  # default unknown

        # --- Committee score: best score across all buying members ---
        sector = (buy_rows[0].get("issuer_sector") or "").lower()
        comm_score = max(
            _committee_score(r.get("committees"), sector) for r in buy_rows
        )

        mc_bonus = _small_cap_bonus(market_caps.get(ticker))
        sell_penalty = _sell_pressure(len(sell_ids), unique_buyers)
        strategic, disruption = llm_scores.get(ticker, (0.0, 0.0))
        total = comm_score + track + cluster + opt_bonus + size + mc_bonus + sell_penalty + strategic + disruption

        pub_dates = [r["pub_date"] for r in ticker_rows]
        tx_dates  = [r["tx_date"] for r in buy_rows if r["tx_date"]]

        # Collect buyer details for display
        buyers_detail = []
        seen_buyers = set()
        for r in buy_rows:
            if r["member_id"] not in seen_buyers:
                seen_buyers.add(r["member_id"])
                buyers_detail.append({
                    "name": r["member_name"],
                    "party": r["party"],
                    "chamber": r["chamber"],
                    "tier": r["amount_tier"],
                })

        sig = {
            "ticker": ticker,
            "issuer_name": buy_rows[0]["issuer_name"],
            "sector": sector,
            "first_seen": min(pub_dates),
            "last_seen": max(pub_dates),
            "first_tx_date": min(tx_dates) if tx_dates else "",
            "last_tx_date": max(tx_dates) if tx_dates else "",
            "member_count": unique_buyers,
            "sell_count": len(sell_ids),
            "sell_pressure_score": sell_penalty,
            "has_options": bool(option_rows),
            "max_tier": max_tier,
            "committee_score": comm_score,
            "track_record_score": track,
            "cluster_score": cluster,
            "options_bonus": opt_bonus,
            "size_score": size,
            "small_cap_score": mc_bonus,
            "strategic_score": strategic,
            "disruption_score": disruption,
            "total_score": round(total, 2),
            "buyers": buyers_detail,
        }
        signals.append(sig)

    signals.sort(key=lambda x: x["total_score"], reverse=True)

    # Upsert into signals table
    now = datetime.now().isoformat(timespec="seconds")
    for sig in signals:
        conn.execute(
            """
            INSERT INTO signals (
                ticker, issuer_name, first_seen, last_seen, first_tx_date, member_count,
                committee_score, track_record_score, cluster_score,
                options_bonus, size_score, small_cap_score,
                sell_pressure_score, strategic_score, disruption_score, total_score, updated_at
            ) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
            ON CONFLICT(ticker) DO UPDATE SET
                issuer_name         = excluded.issuer_name,
                last_seen           = excluded.last_seen,
                first_tx_date       = excluded.first_tx_date,
                member_count        = excluded.member_count,
                committee_score     = excluded.committee_score,
                track_record_score  = excluded.track_record_score,
                cluster_score       = excluded.cluster_score,
                options_bonus       = excluded.options_bonus,
                size_score          = excluded.size_score,
                small_cap_score     = excluded.small_cap_score,
                sell_pressure_score = excluded.sell_pressure_score,
                strategic_score     = excluded.strategic_score,
                disruption_score    = excluded.disruption_score,
                total_score         = excluded.total_score,
                updated_at          = excluded.updated_at
            """,
            (
                sig["ticker"],
                sig["issuer_name"],
                sig["first_seen"],
                sig["last_seen"],
                sig["first_tx_date"] or None,
                sig["member_count"],
                sig["committee_score"],
                sig["track_record_score"],
                sig["cluster_score"],
                sig["options_bonus"],
                sig["size_score"],
                sig["small_cap_score"],
                sig["sell_pressure_score"],
                sig["strategic_score"],
                sig["disruption_score"],
                sig["total_score"],
                now,
            ),
        )

    conn.commit()
    conn.close()
    return signals

def top_signals(n: int = 10, min_score: float = 0.0) -> list[dict]:
    """Return top N signals from the signals table, optionally filtered by min_score."""
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    rows = conn.execute(
        """
        SELECT * FROM signals
        WHERE total_score >= ?
        ORDER BY total_score DESC
        LIMIT ?
        """,
        (min_score, n),
    ).fetchall()
    conn.close()
    return [dict(r) for r in rows]

if __name__ == "__main__":
    print("Scoring all tickers...\n")
    signals = score_tickers()

    print(f"{'Rank':<5} {'Ticker':<8} {'Score':>6}  {'Clust':>5} {'Size':>4} {'Opts':>4} {'Trck':>4} {'Cmte':>4} {'Cap':>4}  "
          f"{'Buyers':>6} {'Sells':>5}  {'Sector'}")
    print("-" * 100)
    for i, s in enumerate(signals[:20], 1):
        opt_flag = "OPT" if s["has_options"] else ""
        buyers_str = " / ".join(
            f"{b['name'].split()[-1]}({'R' if b['party'] in ('republican', 'R') else 'D'})"
            for b in s["buyers"][:3]
        )
        if len(s["buyers"]) > 3:
            buyers_str += f" +{len(s['buyers'])-3}"
        sc = s.get("small_cap_score", 0) or 0
        print(
            f"{i:<5} {s['ticker']:<8} {s['total_score']:>6.1f}  "
            f"{s['cluster_score']:>5.2f} {s['size_score']:>4.0f} "
            f"{s['options_bonus']:>4.0f} {s['track_record_score']:>4.1f} "
            f"{s['committee_score']:>4.0f} {sc:>+4.1f}  "
            f"{s['member_count']:>6} {s['sell_count']:>5}  "
            f"{s['sector'][:25]}"
        )
        print(f"       {s['issuer_name'][:50]:<50}  buyers: {buyers_str}")
        print()

    notify_candidates = [s for s in signals if s["total_score"] >= SCORE_NOTIFY_THRESHOLD]
    print(f"\n{len(notify_candidates)} signals at or above notify threshold ({SCORE_NOTIFY_THRESHOLD})")

    from tools.generate_report import generate
    generate()


Excellent! Now let me get the notification policy and other key files.


"""Notification gating for high-signal trade ideas."""

from __future__ import annotations

from dataclasses import dataclass
from datetime import datetime, timedelta
import math
import re
import sqlite3

from config import (
    DB_PATH,
    NOTIFY_MAX_SELL_PRESSURE_PENALTY,
    NOTIFY_MIN_CONVICTION,
    NOTIFY_MIN_EDGE_FACTORS,
    NOTIFY_REQUIRE_BUY_ACTION,
    NOTIFY_TICKER_COOLDOWN_DAYS,
    SCORE_NOTIFY_THRESHOLD,
)

@dataclass(frozen=True)
class NotificationDecision:
    allowed: bool
    reason: str
    edge_factors: tuple[str, ...] = ()

_CONVICTION_ORDER = {"LOW": 0, "MEDIUM": 1, "HIGH": 2}
_ACTION_BLOCKLIST = (
    "AVOID",
    "HOLD",
    "MONITOR",
    "PASS",
    "SELL",
    "TRIM",
    "WAIT",
)

def _to_float(value, default: float = 0.0) -> float:
    try:
        return float(value)
    except (TypeError, ValueError):
        return default

def edge_factors(sig: dict, research: dict | None) -> tuple[str, ...]:
    """Return the independent evidence factors that make a signal notification-worthy."""
    factors: list[str] = []

    if int(sig.get("member_count") or 0) >= 2:
        factors.append("multi-member cluster")
    if int(sig.get("max_tier") or 0) >= 4:
        factors.append("large disclosed size")
    if sig.get("has_options"):
        factors.append("options timing")
    if _to_float(sig.get("committee_score")) >= 2.0:
        factors.append("direct committee alignment")
    if _to_float(sig.get("small_cap_score")) >= 2.0:
        factors.append("undercovered small/mid cap")

    strategic = _to_float((research or {}).get("strategic_score", sig.get("strategic_score")))
    disruption = _to_float((research or {}).get("disruption_score", sig.get("disruption_score")))
    if strategic + disruption >= 4.0:
        factors.append("strong LLM catalyst")

    return tuple(factors)

def is_buy_recommendation(action: str | None) -> bool:
    """Accept explicit buy/add actions, not incidental mentions of the word buy."""
    if not action:
        return False

    text = re.sub(r"\s+", " ", action.strip().upper())
    if not text:
        return False
    if "DO NOT BUY" in text or "NOT BUY" in text:
        return False
    if text.startswith(_ACTION_BLOCKLIST):
        return False

    first_word = re.split(r"[\s:;-]+", text, maxsplit=1)[0]
    return first_word in {"BUY", "ADD", "ACCUMULATE", "INITIATE"}

def evaluate_notification_candidate(
    sig: dict,
    research: dict | None,
    adjusted_score: float,
    *,
    notify_threshold: float = SCORE_NOTIFY_THRESHOLD,
    min_conviction: str = NOTIFY_MIN_CONVICTION,
    require_buy_action: bool = NOTIFY_REQUIRE_BUY_ACTION,
    min_edge_factors: int = NOTIFY_MIN_EDGE_FACTORS,
    max_sell_pressure_penalty: float = NOTIFY_MAX_SELL_PRESSURE_PENALTY,
) -> NotificationDecision:
    """Evaluate all quality gates except duplicate/cooldown state."""
    ticker = sig.get("ticker", "?")

    if adjusted_score < notify_threshold:
        return NotificationDecision(False, f"adjusted score {adjusted_score:.1f} below {notify_threshold:.1f}")
    if not research:
        return NotificationDecision(False, "no research")
    if not research.get("risk_factors"):
        return NotificationDecision(False, "no deep research")

    conviction = (research.get("conviction") or "LOW").upper()
    min_order = _CONVICTION_ORDER.get(min_conviction.upper(), 1)
    if _CONVICTION_ORDER.get(conviction, 0) < min_order:
        return NotificationDecision(False, f"conviction={conviction} below {min_conviction.upper()}")

    action = research.get("recommended_action") or ""
    if require_buy_action and not is_buy_recommendation(action):
        return NotificationDecision(False, f"recommended_action is not an explicit buy/add: {action[:80]}")

    sell_pressure = _to_float(sig.get("sell_pressure_score"))
    if sell_pressure < max_sell_pressure_penalty:
        return NotificationDecision(False, f"sell pressure {sell_pressure:+.1f} conflicts with buy signal")

    factors = edge_factors(sig, research)
    if len(factors) < min_edge_factors:
        return NotificationDecision(
            False,
            f"only {len(factors)}/{min_edge_factors} edge factors: {', '.join(factors) or 'none'}",
            factors,
        )

    return NotificationDecision(True, f"{ticker}: {len(factors)} edge factors", factors)

def _parse_dt(value: str | None) -> datetime | None:
    if not value:
        return None
    try:
        return datetime.fromisoformat(value)
    except ValueError:
        try:
            return datetime.fromisoformat(value.replace(" ", "T"))
        except ValueError:
            return None

def recent_ticker_notification(
    ticker: str,
    *,
    db_path: str = DB_PATH,
    cooldown_days: int = NOTIFY_TICKER_COOLDOWN_DAYS,
    now: datetime | None = None,
) -> tuple[bool, datetime | None, int]:
    """
    Return whether ticker is inside the rolling cooldown, plus latest notify time
    and whole days remaining.
    """
    now = now or datetime.now()
    cutoff = now - timedelta(days=cooldown_days)

    conn = sqlite3.connect(db_path)
    rows = conn.execute(
        """
        SELECT notified_at
        FROM signal_outcomes
        WHERE ticker = ?
        ORDER BY notified_at DESC
        LIMIT 10
        """,
        (ticker,),
    ).fetchall()
    conn.close()

    latest = None
    for row in rows:
        parsed = _parse_dt(row[0])
        if parsed and (latest is None or parsed > latest):
            latest = parsed

    if latest is None or latest < cutoff:
        return False, latest, 0

    remaining_seconds = (latest + timedelta(days=cooldown_days) - now).total_seconds()
    days_remaining = max(1, math.ceil(remaining_seconds / 86400))
    return True, latest, days_remaining


Now let me get the config file and the README to understand better.


import os
from dotenv import load_dotenv
load_dotenv()

# --- Database ---
DB_PATH = os.path.join(os.path.dirname(__file__), "data", "congress_trades.db")

# --- Ingest ---
LOOKBACK_DAYS = 30          # how many days of trade history to fetch on each run
PAGE_SIZE = 100             # max records per page from Capitol Trades

# --- Scoring ---
SCORE_RESEARCH_THRESHOLD = 5.0   # min score to trigger LLM research
SCORE_NOTIFY_THRESHOLD = 9.0     # min score to send ntfy notification

# --- Timeliness multipliers applied to score at notify-decision time only ---
# A "late" signal's stored score is unchanged; the multiplier only gates notifications.
TIMELINESS_MULTIPLIERS = {"fresh": 1.0, "watch": 0.85, "late": 0.70}

# --- Win rate ---
WIN_RATE_MIN_TRADES = 3          # minimum buy trades with price data before using actual rate

# --- Deep Research ---
# All signals above SCORE_NOTIFY_THRESHOLD are eligible for deep research.
# Each run processes up to TOP_N of them, prioritising never-researched tickers
# then stalest-first, skipping any researched within DECAY_DAYS.
# The decay window (not TOP_N) is the primary cost control.
DEEP_RESEARCH_TOP_N = 10            # max deep-research calls per run
DEEP_RESEARCH_DECAY_DAYS = 7        # days before re-researching the same ticker

# --- Notification quality gates ---
# Cap how many push alerts fire per pipeline run regardless of score.
# Push should be reserved for the single best actionable idea; the report carries the rest.
MAX_NOTIFICATIONS_PER_RUN = 1

# Do not re-alert the same ticker during this rolling window unless --force is used.
NOTIFY_TICKER_COOLDOWN_DAYS = 14

# When True, only signals where deep research recommends "BUY NOW" or "BUY" get notified.
# Set False to notify for MONITOR/AVOID signals too (not recommended — defeats the purpose).
NOTIFY_REQUIRE_BUY_ACTION = True

# Minimum conviction level to trigger notification. Options: "LOW", "MEDIUM", "HIGH"
NOTIFY_MIN_CONVICTION = "HIGH"

# Minimum number of independent edge factors before a signal can page you.
# Factors include multi-member clusters, large disclosed size, options timing,
# committee alignment, small/mid-cap scarcity, and strong LLM catalyst scores.
NOTIFY_MIN_EDGE_FACTORS = 3

# Suppress buy notifications when sellers materially offset buyers.
NOTIFY_MAX_SELL_PRESSURE_PENALTY = -1.0

# --- LLM ---
LLM_PROVIDER = "anthropic"          # "anthropic" or "google"
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY", "")
GOOGLE_API_KEY = os.getenv("GOOGLE_API_KEY", "")

# --- Predictive Signals ---
# Behavioral analysis: predict future trades before disclosure.
# Requires CONGRESS_GOV_API_KEY for live bill data (free at api.congress.gov);
# works in degraded mode (LLM-only) without it.
CONGRESS_GOV_API_KEY = os.getenv("CONGRESS_GOV_API_KEY", "")
PREDICTOR_MAX_MEMBERS = 5          # members to analyze per run (cost control)
PREDICTOR_DECAY_DAYS  = 7          # skip re-predicting same member within this window
PREDICTOR_MIN_TRADES  = 5          # min historical trades before a member is analyzed
PREDICTOR_LEGISLATION_DAYS = 45   # how far back to look for recently active bills

# --- ntfy ---
NTFY_BASE_URL = "https://ntfy.sh"
NTFY_TOPIC_SIGNALS = "congressWatchSignals"        # trade signal notifications
NTFY_TOPIC_STATUS = "claudeCodeWaitingOnYou"       # system status / needs-input alerts


Great! Now let me get the README and the schema to understand data persistence.


# CongressWatch

Autonomous congressional trade signal system. Ingests STOCK Act disclosures, scores them for signal quality, researches top candidates via LLM, and sends push notifications when conviction crosses a threshold.

## Why

Congressional members are legally required to disclose trades over $1,000 within 45 days (STOCK Act, 2012). Several members have shown consistent outperformance. Existing trackers (Capitol Trades, Unusual Whales, Autopilot) surface the raw data but do no analysis. This system adds:

- **Committee-weighted scoring** — trades by members on relevant oversight committees carry more weight
- **Cluster detection** — multiple members buying the same ticker simultaneously amplifies signal, discounted for mega-caps routinely held by many members
- **Small-cap boost** — smaller companies have less analyst coverage; specific catalysts move them more in percentage terms
- **LLM research** — autonomous investigation of *why* a trade happened and whether the thesis is still timely; high-conviction signals get a deeper bear-case analysis
- **Price performance tracking** — return since member's trade date vs. return since first disclosure, surfacing the information-decay gap
- **Conviction-filtered notifications** — push alerts require score, deep research, explicit buy action, multiple edge factors, and a rolling ticker cooldown

## Quick Start

```bash
# Set up environment
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt

# Copy and populate your API keys
cp .env.example .env   # edit with your keys

# Load 1 year of history (~5–10 min, safe to re-run)
.venv/bin/python tools/backfill.py

# Full pipeline: ingest → committees → score → prices → research → notify
.venv/bin/python run_daily.py
```

## Running

```bash
# Full pipeline
.venv/bin/python run_daily.py

# Skip ingest (re-score + research + notify only — useful for testing)
.venv/bin/python run_daily.py --no-ingest

# Dry run — score and report only, no LLM calls, no notifications
.venv/bin/python run_daily.py --dry-run

# Re-notify signals already sent today (useful after tuning thresholds)
.venv/bin/python run_daily.py --force

# Score only and regenerate report (fast, no network)
.venv/bin/python analysis/scorer.py

# Fetch/refresh prices and market caps
.venv/bin/python tools/price_fetcher.py

# CLI signal table
.venv/bin/python tools/report.py
```

The markdown report is always written to `reports/signals_latest.md` after scoring.

## Pipeline

Each daily run executes six steps in order:

1. **Ingest** — scrape Capitol Trades for new disclosures; upsert members and trades into SQLite
2. **Committees** — refresh member committee assignments from `congress-legislators` (GitHub, weekly cache); committee data updates automatically when a new Congress is seated
3. **Score** — compute signal scores for all tickers with a buy in the lookback window; regenerate report
4. **Prices** — fetch yesterday's close and market cap via yfinance for all research-eligible signals; compute return since member's trade and return since first disclosure
5. **Research** — call Claude API for signals above `SCORE_RESEARCH_THRESHOLD`; signals at/above `SCORE_NOTIFY_THRESHOLD` get a deeper prompt with explicit `risk_factors` and `recommended_action`
6. **Notify** — send at most one ntfy push for the strongest notify-threshold signal that passes high-conviction quality gates and is outside the ticker cooldown

## Scoring System

```
total_score = committee + track_record + cluster × breadth_discount + options_bonus + size + small_cap
```

| Component | Range | Logic |
|---|---|---|
| `committee_score` | 0–3 | Member's committee has jurisdiction over the company's sector (e.g. Armed Services + defense contractor = 3) |
| `track_record_score` | 0–2 | Historical win rate; defaults to 1.0 until enough data accumulates |
| `cluster_score` | 0–3 × discount | 2 buyers=1, 3=2, 4+=3; multiplied by breadth discount (0.4–1.0) based on how many members have historically bought this ticker |
| `options_bonus` | 0 or 2 | Options trade on this ticker (+2 — implies specific timing knowledge) |
| `size_score` | 0–5 | Highest dollar tier among buys: $1K-$15K=1, $15K-$50K=2, $50K-$100K=3, $100K-$250K=4, $250K+=5 |
| `small_cap_score` | −1 to +3 | Market cap tier: mega >$100B = −1, large $10–100B = 0, mid $2–10B = +1.5, small <$2B = +3 |

**Breadth discount** — tickers routinely bought by many members across history are portfolio staples, not signals. Cluster score is multiplied by: 0.4 (≥20 historical buyers), 0.65 (≥12), 0.85 (≥6), 1.0 (<6).

**Notify threshold**: `≥ 9.0` (configurable in `config.py` as `SCORE_NOTIFY_THRESHOLD`)
**Research threshold**: `≥ 5.0` (configurable as `SCORE_RESEARCH_THRESHOLD`)

## Notification Gate

Push alerts are intentionally stricter than report inclusion. A signal must pass all of these checks:

- Raw score at or above `SCORE_NOTIFY_THRESHOLD`, after timeliness adjustment.
- Deep research exists, including `risk_factors`.
- `conviction` is at least `NOTIFY_MIN_CONVICTION` (`HIGH` by default).
- `recommended_action` starts with an explicit buy/add action, not merely a sentence containing “buy.”
- Sell pressure is not meaningfully conflicting (`sell_pressure_score >= -1.0` by default).
- At least `NOTIFY_MIN_EDGE_FACTORS` independent edge factors are present: multi-member cluster, large disclosed size, options timing, direct committee alignment, undercovered small/mid cap, or strong LLM catalyst score.
- The ticker has not already fired within `NOTIFY_TICKER_COOLDOWN_DAYS` (`14` by default), unless `--force` is used.

`MAX_NOTIFICATIONS_PER_RUN` defaults to `1`; the HTML/markdown report is the review surface for additional candidates.

## Notification Format

```
CongressWatch: $DMLP (score 9.0)
Dorchester Minerals LP — 1 member bought
Size: $15K-$50K
Timeliness: FRESH | Conviction: HIGH

[company summary]

Catalyst: [why this member with these committees would buy now]

→ BUY NOW — [one-sentence rationale]
```

## Data Source

**Primary**: Capitol Trades (capitoltrades.com) — free, no API key, updated within hours of new filings. Data parsed directly from React server components in page HTML.

**Committees**: `unitedstates/congress-legislators` GitHub repo — `committee-membership-current.yaml`. Auto-refreshes weekly; picks up new Congress members automatically.

**Prices**: yfinance — yesterday's close, no API key required.

Typical disclosure lag: 30–75 days after trade execution (STOCK Act allows 45 days to file). The `TxRet%` column in the report shows return since the member's trade date; `PubRet%` shows return since first disclosure — the gap between them is the information value that decayed before it became public.

## Directory Structure

```
config.py                  # thresholds, API keys, ntfy topic
run_daily.py               # main entry point

ingest/
  capitol_trades.py        # Capitol Trades HTML scraper + SQLite writer
  committees.py            # weekly committee assignment refresh

analysis/
  scorer.py                # signal scoring engine (runs → regenerates report)
  researcher.py            # LLM research pipeline; deep prompt for notify-threshold signals
  timeliness.py            # age-based freshness assessment
  llm_client.py            # provider abstraction (Anthropic / Google)

notify/
  ntfy_client.py           # push notifications via ntfy.sh

tools/
  report.py                # CLI: scored signals + recent trades
  generate_report.py       # writes reports/signals_latest.md
  price_fetcher.py         # yfinance price + market cap fetch
  backfill.py              # load 1–2 years of history

db/
  schema.sql               # table definitions
  migrate.py               # additive column migrations (safe to re-run)

data/
  congress_trades.db       # SQLite (gitignored)

reports/
  signals_latest.md        # always current after scoring
  signals_YYYY-MM-DD.md    # dated snapshots
  token_usage.log          # LLM cost tracking

viewer/
  app.py                   # Flask dashboard (port 5001)
```

## Configuration

Edit `config.py` or set environment variables in `.env`:

```python
LOOKBACK_DAYS = 30                  # how far back to ingest on each run
SCORE_NOTIFY_THRESHOLD = 9.0        # minimum score before notification gates
SCORE_RESEARCH_THRESHOLD = 5.0      # minimum score to trigger LLM research
MAX_NOTIFICATIONS_PER_RUN = 1       # cap push alerts per run
NOTIFY_TICKER_COOLDOWN_DAYS = 14    # suppress repeated same-ticker alerts
NOTIFY_MIN_CONVICTION = "HIGH"      # LOW / MEDIUM / HIGH
NOTIFY_MIN_EDGE_FACTORS = 3         # independent evidence factors required
LLM_PROVIDER = "anthropic"          # "anthropic" or "google"
ANTHROPIC_API_KEY = os.getenv("ANTHROPIC_API_KEY", "")
NTFY_TOPIC_SIGNALS = "congressWatchSignals"
```

## Cron / Scheduling

**Mac (launchd)**:
```bash
# Run Mon–Fri at 6am — place in ~/Library/LaunchAgents/
# See: https://www.launchd.info for plist format
```

**Linux / WSL2 (cron)**:
```bash
crontab -e
# Add:
0 6 * * 1-5  /home/james/trading_ideas/.venv/bin/python /home/james/trading_ideas/run_daily.py

# WSL2: ensure cron service is running
sudo service cron start
# To auto-start, add to /etc/wsl.conf:
# [boot]
# command = service cron start
```

## Data Notes

- Dollar amounts are **ranges**, not exact (STOCK Act only requires range disclosure). `value_estimate` is the midpoint; `amount_tier` is the derived 1–6 tier.
- Options trades are disclosed without strike price or expiration date.
- `issuer_ticker` is blank for some positions (mutual funds, ETFs without a standard ticker, foreign-listed names).
- Market cap is fetched once per daily run and cached; the small-cap scoring component is applied on the *next* scoring run (market cap does not change meaningfully day to day).
- The first full pipeline run on a new machine produces `small_cap_score = 0` for all signals — it populates on the second run once prices/caps are fetched.


Now let me fetch the database schema and the researcher.py file to understand the research and "shadow mode" concept.


CREATE TABLE IF NOT EXISTS members (
    bioguide_id     TEXT PRIMARY KEY,
    first_name      TEXT,
    last_name       TEXT,
    nickname        TEXT,
    party           TEXT,
    chamber         TEXT,
    state_id        TEXT,
    gender          TEXT,
    dob             TEXT,
    committees      TEXT,   -- JSON array, refreshed weekly
    trade_count     INTEGER DEFAULT 0,
    win_rate        REAL,   -- nullable until calculable
    updated_at      TEXT
);

CREATE TABLE IF NOT EXISTS trades (
    tx_id           INTEGER PRIMARY KEY,
    member_id       TEXT NOT NULL,
    issuer_id       INTEGER,
    issuer_name     TEXT,
    issuer_ticker   TEXT,   -- clean ticker, e.g. "NVDA"
    issuer_sector   TEXT,
    issuer_country  TEXT,
    chamber         TEXT,
    tx_date         TEXT,   -- YYYY-MM-DD, when trade executed
    pub_date        TEXT,   -- when disclosure was published
    reporting_gap   INTEGER, -- days between tx_date and pub_date
    tx_type         TEXT,   -- buy / sell / exchange
    tx_type_extended TEXT,
    owner           TEXT,   -- self / spouse / not-disclosed / etc
    price_at_trade  REAL,
    value_estimate  INTEGER, -- midpoint dollar estimate from disclosure
    amount_tier     INTEGER, -- 1-6 derived from value_estimate
    is_option       INTEGER DEFAULT 0,  -- 0/1
    comment         TEXT,
    source          TEXT DEFAULT 'capitol_trades',
    created_at      TEXT DEFAULT (datetime('now'))
);

CREATE TABLE IF NOT EXISTS signals (
    id                  INTEGER PRIMARY KEY AUTOINCREMENT,
    ticker              TEXT NOT NULL UNIQUE,
    first_seen          TEXT,
    last_seen           TEXT,
    first_tx_date       TEXT,   -- earliest trade exec date in current scoring window
    member_count        INTEGER DEFAULT 1,
    committee_score     REAL DEFAULT 0,
    track_record_score  REAL DEFAULT 1,
    cluster_score       REAL DEFAULT 0,
    options_bonus       REAL DEFAULT 0,
    size_score          REAL DEFAULT 0,
    total_score         REAL DEFAULT 0,
    research_summary    TEXT,
    timeliness          TEXT,   -- fresh / watch / late
    price_at_tx         REAL,   -- close on first_tx_date (member's entry)
    price_at_pub        REAL,   -- close on first_seen (our actionable entry)
    price_current       REAL,   -- yesterday's close
    return_from_tx_pct  REAL,   -- % gain/loss since member's trade
    return_from_pub_pct REAL,   -- % gain/loss since first disclosure
    price_fetched_at    TEXT,
    market_cap          REAL,   -- cached from yfinance
    small_cap_score     REAL DEFAULT 0,  -- score component: bonus for smaller caps
    sell_pressure_score REAL DEFAULT 0,  -- score component: penalty when sellers ≥ buyers
    notified_at         TEXT,
    notified            INTEGER DEFAULT 0,
    updated_at          TEXT DEFAULT (datetime('now'))
);

CREATE TABLE IF NOT EXISTS research_cache (
    ticker              TEXT PRIMARY KEY,
    cached_at           TEXT,
    deep_researched_at  TEXT,   -- set when deep prompt (risk_factors + recommended_action) was run
    company_summary     TEXT,
    recent_news         TEXT,
    catalyst_hypothesis TEXT,
    timeliness_assessment TEXT,
    conviction          TEXT,   -- LOW / MEDIUM / HIGH
    conviction_rationale TEXT,
    risk_factors        TEXT,
    recommended_action  TEXT,
    raw_response        TEXT
);

CREATE TABLE IF NOT EXISTS signal_outcomes (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    ticker          TEXT NOT NULL,
    notified_at     TEXT NOT NULL,
    score_at_notify REAL,
    conviction      TEXT,
    recommended_action TEXT,
    price_at_notify REAL,
    price_30d       REAL,
    price_60d       REAL,
    return_30d_pct  REAL,
    return_60d_pct  REAL,
    outcome         TEXT,   -- "win" / "loss" / "pending"
    checked_at      TEXT,
    created_at      TEXT DEFAULT (datetime('now'))
);

CREATE TABLE IF NOT EXISTS predicted_signals (
    id                      INTEGER PRIMARY KEY AUTOINCREMENT,
    member_id               TEXT NOT NULL,
    member_name             TEXT,
    ticker                  TEXT NOT NULL,
    company_name            TEXT,
    sector                  TEXT,
    predicted_action        TEXT,   -- buy / sell
    confidence              TEXT,   -- LOW / MEDIUM / HIGH
    legislation_catalyst    TEXT,
    historical_precedent    TEXT,
    rationale               TEXT,
    behavioral_profile      TEXT,   -- member's trading style summary (one per member)
    generated_at            TEXT,
    confirmed               INTEGER DEFAULT 0,  -- 1 when a matching real disclosure appears
    confirmed_at            TEXT,
    UNIQUE(member_id, ticker)
);

CREATE INDEX IF NOT EXISTS idx_trades_member ON trades(member_id);
CREATE INDEX IF NOT EXISTS idx_trades_ticker ON trades(issuer_ticker);
CREATE INDEX IF NOT EXISTS idx_trades_pub_date ON trades(pub_date);
CREATE INDEX IF NOT EXISTS idx_trades_tx_date ON trades(tx_date);
CREATE INDEX IF NOT EXISTS idx_outcomes_ticker ON signal_outcomes(ticker);
CREATE INDEX IF NOT EXISTS idx_outcomes_notified_at ON signal_outcomes(notified_at);
CREATE INDEX IF NOT EXISTS idx_predicted_member ON predicted_signals(member_id);
CREATE INDEX IF NOT EXISTS idx_predicted_ticker ON predicted_signals(ticker);

CREATE TABLE IF NOT EXISTS ticker_metadata (
    ticker          TEXT PRIMARY KEY,
    market_cap      REAL,
    last_price      REAL,
    is_etf          INTEGER DEFAULT 0,   -- 1 if yfinance quoteType is ETF/MUTUALFUND
    is_valid        INTEGER DEFAULT 1,   -- 0 if yfinance returns no data (invalid/delisted)
    pe_ratio        REAL,                -- trailing P/E
    analyst_rating  TEXT,                -- strong buy / buy / hold / sell / strong sell
    price_vs_50d_pct REAL,              -- % current price above/below 50-day MA
    fetched_at      TEXT
);

CREATE INDEX IF NOT EXISTS idx_ticker_meta ON ticker_metadata(ticker);


Excellent! Now let me get the researcher.py to understand what "deep research" does.


curl -s "https://raw.githubusercontent.com/jamesjlopez/congressional-trading/main/analysis/researcher.py" | head -600


"""
LLM research pipeline.

For each signal above SCORE_RESEARCH_THRESHOLD:
  1. Gather context from DB (company, buyers, committees, trade dates)
  2. Build a structured research prompt
  3. Call LLM and parse the response
  4. Store result in research_cache; update signal timeliness
  5. Log token usage to reports/token_usage.log

Usage:
  python analysis/researcher.py              # research all eligible signals
  python analysis/researcher.py --dry-run    # print prompts, no API call
  python analysis/researcher.py --ticker MSFT
"""

from __future__ import annotations
import sqlite3
import sys
import os
import json
import argparse
from datetime import datetime, timedelta

sys.path.insert(0, os.path.dirname(os.path.dirname(__file__)))
from config import (DB_PATH, SCORE_RESEARCH_THRESHOLD, SCORE_NOTIFY_THRESHOLD,
                    LOOKBACK_DAYS, DEEP_RESEARCH_TOP_N, DEEP_RESEARCH_DECAY_DAYS)
from analysis.timeliness import assess_age, age_days

CACHE_TTL_HOURS = 24
TOKEN_LOG = os.path.join(os.path.dirname(os.path.dirname(__file__)), "reports", "token_usage.log")

SYSTEM_PROMPT = (
    "You are a concise financial research analyst specializing in US congressional "
    "trading activity. Provide factual, evidence-based analysis. Be specific about "
    "dates, dollar amounts, and named legislation or contracts where relevant. "
    "Keep each section brief — 2-4 sentences maximum. Do not speculate beyond "
    "publicly available information."
)

RESEARCH_PROMPT_TEMPLATE = """\
A member (or members) of the US Congress recently disclosed a stock purchase in \
{ticker} ({company_name}, sector: {sector}).

TRADE DETAILS:
{buyer_lines}

Most recent disclosure: {last_seen} (trade executed: {last_tx_date})
Signal age: {age_days} days since last disclosure
Highest dollar tier: {tier_label}
Options activity: {options_flag}

Please research this and respond in the following JSON format exactly:

{{
  "company_summary": "2-3 sentence description of what this company does and its market position.",
  "recent_news": "Key news from the last 90 days relevant to this company — earnings, contracts, regulatory actions, M&A.",
  "catalyst_hypothesis": "Why might a congress member with these committee assignments have bought this now? Reference specific legislation, contracts, or policy areas if applicable.",
  "timeliness": "fresh|watch|late — is the thesis still intact given the signal is {age_days} days old? Explain in one sentence.",
  "conviction": "LOW|MEDIUM|HIGH",
  "conviction_rationale": "One sentence explaining the conviction level.",
  "strategic_score": <integer 0-3>,
  "strategic_rationale": "One sentence. Score guide — 0: no national-security/infrastructure nexus; 1: indirect sector relevance; 2: defense/energy/critical-infrastructure contractor with known government exposure; 3: direct military, dual-use, or critical-infrastructure application that Congress would have privileged visibility into.",
  "disruption_score": <integer 0-3>,
  "disruption_rationale": "One sentence. Score guide — 0: incremental improvement to existing market; 1: meaningful differentiation; 2: significant market-shift potential; 3: category-creating or paradigm-shifting technology/business model."
}}

Respond with valid JSON only, no markdown fencing."""

DEEP_RESEARCH_PROMPT_TEMPLATE = """\
A member (or members) of the US Congress recently disclosed a stock purchase in \
{ticker} ({company_name}, sector: {sector}).

TRADE DETAILS:
{buyer_lines}

Most recent disclosure: {last_seen} (trade executed: {last_tx_date})
Signal age: {age_days} days since last disclosure
Highest dollar tier: {tier_label}
Options activity: {options_flag}

This signal scored above the high-conviction notify threshold. Please provide \
deeper analysis including explicit bear-case considerations. Respond in the \
following JSON format exactly:

{{
  "company_summary": "2-3 sentence description of what this company does and its market position.",
  "recent_news": "Key news from the last 90 days relevant to this company — earnings, contracts, regulatory actions, M&A.",
  "catalyst_hypothesis": "Why might a congress member with these committee assignments have bought this now? Reference specific legislation, contracts, or policy areas if applicable.",
  "timeliness": "fresh|watch|late — is the thesis still intact given the signal is {age_days} days old? Explain in one sentence.",
  "conviction": "LOW|MEDIUM|HIGH",
  "conviction_rationale": "One sentence explaining the conviction level.",
  "risk_factors": "What would invalidate this thesis? Name 2-3 specific risks: regulatory, earnings, sector headwinds, or that the position is already fully priced in.",
  "recommended_action": "BUY NOW|MONITOR|AVOID — one sentence rationale for the explicit recommendation.",
  "strategic_score": <integer 0-3>,
  "strategic_rationale": "One sentence. Score guide — 0: no national-security/infrastructure nexus; 1: indirect sector relevance; 2: defense/energy/critical-infrastructure contractor with known government exposure; 3: direct military, dual-use, or critical-infrastructure application that Congress would have privileged visibility into.",
  "disruption_score": <integer 0-3>,
  "disruption_rationale": "One sentence. Score guide — 0: incremental improvement to existing market; 1: meaningful differentiation; 2: significant market-shift potential; 3: category-creating or paradigm-shifting technology/business model."
}}

Respond with valid JSON only, no markdown fencing."""

_TIER_LABELS = {
    0: "unknown",
    1: "$1K–$15K",
    2: "$15K–$50K",
    3: "$50K–$100K",
    4: "$100K–$250K",
    5: "$250K–$500K",
    6: "$500K+",
}

def _build_prompt(sig: dict, buyers: list[dict], deep: bool = False) -> str:
    buyer_lines = []
    for b in buyers:
        committees = ""
        if b.get("committees"):
            try:
                clist = json.loads(b["committees"])
                if clist:
                    committees = f", committees: {', '.join(str(c) for c in clist[:3])}"
            except Exception:
                pass
        buyer_lines.append(
            f"  - {b['member_name']} ({b['party'][:1].upper()}, {b['chamber']})"
            f", tier {b['amount_tier']}{committees}"
        )

    template = DEEP_RESEARCH_PROMPT_TEMPLATE if deep else RESEARCH_PROMPT_TEMPLATE
    return template.format(
        ticker=sig["ticker"],
        company_name=sig["issuer_name"] or sig["ticker"],
        sector=sig["sector"] or "unknown",
        buyer_lines="\n".join(buyer_lines) or "  (no buyer details)",
        last_seen=sig["last_seen"],
        last_tx_date=sig.get("last_tx_date", "unknown"),
        age_days=age_days(sig["last_seen"]),
        tier_label=_TIER_LABELS.get(sig.get("max_tier", 0), "unknown"),
        options_flag="YES" if sig.get("has_options") else "no",
    )

def _is_cache_valid(cached_at: str) -> bool:
    if not cached_at:
        return False
    try:
        dt = datetime.fromisoformat(cached_at)
        return (datetime.now() - dt).total_seconds() < CACHE_TTL_HOURS * 3600
    except ValueError:
        return False

def _is_deep_cache_valid(deep_researched_at: str, decay_days: int) -> bool:
    """Return True if deep research was done within decay_days."""
    if not deep_researched_at:
        return False
    try:
        dt = datetime.fromisoformat(deep_researched_at)
        return (datetime.now() - dt).days < decay_days
    except ValueError:
        return False

def _log_tokens(ticker: str, usage: dict):
    os.makedirs(os.path.dirname(TOKEN_LOG), exist_ok=True)
    with open(TOKEN_LOG, "a") as f:
        total_cost_est = (
            usage.get("input_tokens", 0) * 3e-6
            + usage.get("output_tokens", 0) * 15e-6
            - usage.get("cache_read_tokens", 0) * 2.7e-6
        )
        f.write(
            f"{datetime.now().isoformat(timespec='seconds')} | {ticker:<8} | "
            f"in={usage.get('input_tokens',0):>5} out={usage.get('output_tokens',0):>4} "
            f"cache_hit={usage.get('cache_read_tokens',0):>5} "
            f"est_cost=${total_cost_est:.4f}\n"
        )

def _parse_response(text: str) -> dict:
    """Extract JSON from LLM response, tolerating minor formatting noise."""
    text = text.strip()
    # Strip markdown code fences if present
    if text.startswith("```"):
        lines = text.splitlines()
        text = "\n".join(lines[1:-1] if lines[-1].strip() == "```" else lines[1:])
    try:
        return json.loads(text)
    except json.JSONDecodeError:
        # Try to find the JSON object within the text
        start = text.find("{")
        end = text.rfind("}") + 1
        if start >= 0 and end > start:
            return json.loads(text[start:end])
        raise

def research_signal(sig: dict, buyers: list[dict], dry_run: bool = False,
                    force_deep: bool = False) -> dict | None:
    """
    Research one signal. Returns parsed result dict or None on failure.
    If dry_run=True, prints the prompt and returns None without calling the LLM.
    If force_deep=True, always uses the deep prompt and skips the regular cache.
    """
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row

    deep = force_deep or (sig.get("total_score", 0) >= SCORE_NOTIFY_THRESHOLD)

    if not force_deep:
        # Regular 24-hour cache check
        cached = conn.execute(
            "SELECT * FROM research_cache WHERE ticker = ?", (sig["ticker"],)
        ).fetchone()
        cache_valid = cached and _is_cache_valid(cached["cached_at"])
        if cache_valid and deep and not cached["risk_factors"]:
            cache_valid = False  # force re-research with deep prompt
        if cache_valid:
            print(f"  {sig['ticker']}: using cached result ({cached['cached_at'][:16]})")
            conn.close()
            return dict(cached)

    prompt = _build_prompt(sig, buyers, deep=deep)

    if dry_run:
        print(f"\n{'='*60}")
        print(f"DRY RUN — would research: {sig['ticker']} (score={sig['total_score']})")
        print(f"{'='*60}")
        print(prompt)
        conn.close()
        return None

    # Call LLM
    from analysis.llm_client import call, LLMError
    try:
        print(f"  {sig['ticker']}: calling LLM...", end=" ", flush=True)
        text, usage = call(prompt, system=SYSTEM_PROMPT)
        _log_tokens(sig["ticker"], usage)
        print(f"done (in={usage['input_tokens']} out={usage['output_tokens']})")
    except LLMError as e:
        print(f"  {sig['ticker']}: LLM error — {e}")
        conn.close()
        return None
    except Exception as e:
        # Catch quota / network errors from underlying SDKs
        msg = str(e)
        if "429" in msg or "RESOURCE_EXHAUSTED" in msg or "quota" in msg.lower():
            print(f"\n  {sig['ticker']}: API quota exceeded.")
            print("  To fix: enable billing at console.cloud.google.com (Google)")
            print("          or set ANTHROPIC_API_KEY and change LLM_PROVIDER='anthropic' in config.py")
        else:
            print(f"  {sig['ticker']}: unexpected error — {e}")
        conn.close()
        return None

    # Parse response
    try:
        result = _parse_response(text)
    except Exception as e:
        print(f"  {sig['ticker']}: parse error — {e}")
        print(f"  Raw response: {text[:200]}")
        conn.close()
        return None

    now = datetime.now().isoformat(timespec="seconds")

    strategic_score  = int(result.get("strategic_score")  or 0)
    disruption_score = int(result.get("disruption_score") or 0)

    # Upsert into research_cache
    conn.execute(
        """
        INSERT INTO research_cache (ticker, cached_at, company_summary, recent_news,
            catalyst_hypothesis, timeliness_assessment, conviction, conviction_rationale,
            risk_factors, recommended_action,
            strategic_score, strategic_rationale, disruption_score, disruption_rationale,
            raw_response)
        VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)
        ON CONFLICT(ticker) DO UPDATE SET
            cached_at             = excluded.cached_at,
            company_summary       = excluded.company_summary,
            recent_news           = excluded.recent_news,
            catalyst_hypothesis   = excluded.catalyst_hypothesis,
            timeliness_assessment = excluded.timeliness_assessment,
            conviction            = excluded.conviction,
            conviction_rationale  = excluded.conviction_rationale,
            risk_factors          = excluded.risk_factors,
            recommended_action    = excluded.recommended_action,
            strategic_score       = excluded.strategic_score,
            strategic_rationale   = excluded.strategic_rationale,
            disruption_score      = excluded.disruption_score,
            disruption_rationale  = excluded.disruption_rationale,
            raw_response          = excluded.raw_response
        """,
        (
            sig["ticker"],
            now,
            result.get("company_summary", ""),
            result.get("recent_news", ""),
            result.get("catalyst_hypothesis", ""),
            result.get("timeliness", "") + " — " + result.get("conviction_rationale", ""),
            result.get("conviction", ""),
            result.get("conviction_rationale", ""),
            result.get("risk_factors") or None,
            result.get("recommended_action") or None,
            strategic_score,
            result.get("strategic_rationale") or None,
            disruption_score,
            result.get("disruption_rationale") or None,
            text,
        ),
    )

    # Write scores back to signals table so scorer can pick them up next run
    conn.execute(
        """
        UPDATE signals
        SET strategic_score  = ?,
            disruption_score = ?,
            total_score      = total_score
                               - COALESCE(strategic_score, 0)
                               - COALESCE(disruption_score, 0)
                               + ? + ?
        WHERE ticker = ?
        """,
        (strategic_score, disruption_score,
         strategic_score, disruption_score,
         sig["ticker"]),
    )

    # Stamp deep_researched_at when the deep prompt was used
    if deep:
        conn.execute(
            "UPDATE research_cache SET deep_researched_at = ? WHERE ticker = ?",
            (now, sig["ticker"]),
        )

    # Update signal timeliness field
    timeliness = result.get("timeliness", assess_age(sig["last_seen"]))
    if "|" in timeliness:
        timeliness = timeliness.split("|")[0].strip()
    conn.execute(
        "UPDATE signals SET timeliness = ?, research_summary = ?, updated_at = ? WHERE ticker = ?",
        (
            timeliness,
            result.get("company_summary", "")[:500],
            now,
            sig["ticker"],
        ),
    )
    conn.commit()
    conn.close()
    return result

def research_top_n_deep(signals: list[dict], top_n: int = DEEP_RESEARCH_TOP_N,
                        decay_days: int = DEEP_RESEARCH_DECAY_DAYS,
                        dry_run: bool = False) -> list[tuple]:
    """
    Deep-research up to top_n signals that are above the notify threshold and
    whose deep research is absent or older than decay_days.

    Prioritisation order (within the eligible set):
      1. Never deep-researched (deep_researched_at IS NULL)
      2. Stalest deep research first (oldest deep_researched_at)
      Both groups are then sorted by total_score descending as a tiebreaker.

    This ensures newly-high-scoring tickers are not starved behind a wall of
    recently-researched high scorers — they jump to the front of the queue.

    Returns list of (sig, result) pairs for every signal actually researched.
    """
    conn = sqlite3.connect(DB_PATH)
    conn.row_factory = sqlite3.Row
    cutoff = (datetime.now() - timedelta(days=LOOKBACK_DAYS)).strftime("%Y-%m-%d")

    # Deep research is only meaningful for signals above the notification threshold.
    # Filter defensively here even if the caller already pre-filtered.
    signals = [s for s in signals if s.get("total_score", 0) >= SCORE_NOTIFY_THRESHOLD]

    # Fetch deep_researched_at for all candidate tickers in one query
    tickers = [s["ticker"] for s in signals]
    if not tickers:
        conn.close()
        return []

    placeholders = ",".join("?" * len(tickers))
    cache_rows = conn.execute(
        f"SELECT ticker, deep_researched_at FROM research_cache WHERE ticker IN ({placeholders})",
        tickers,
    ).fetchall()
    last_deep: dict[str, str | None] = {r["ticker"]: r["deep_researched_at"] for r in cache_rows}

    # Split into eligible (stale/never) and skipped (fresh)
    never: list[dict] = []
    stale: list[dict] = []
    skipped: list[str] = []

    for sig in signals:
        ticker = sig["ticker"]
        dr_at = last_deep.get(ticker)
        if dr_at and _is_deep_cache_valid(dr_at, decay_days):
            skipped.append(f"{ticker} ({dr_at[:10]})")
        elif dr_at is None:
            never.append(sig)
        else:
            stale.append(sig)

    # Sort each group by score descending, then combine: never → stale
    never.sort(key=lambda s: s["total_score"], reverse=True)
    stale.sort(key=lambda s: s["total_score"], reverse=True)
    candidates = never + stale

    if skipped:
        print(f"  Skipped (fresh within {decay_days}d): {', '.join(skipped)}")
    if never:
        print(f"  Never deep-researched: {', '.join(s['ticker'] for s in never)}")

    researched: list[tuple] = []
    for sig in candidates:
        if len(researched) >= top_n:
            break

        ticker = sig["ticker"]
        buyers = conn.execute(
            """
            SELECT DISTINCT t.member_id,
                m.first_name || ' ' || m.last_name AS member_name,
                m.party, m.chamber, m.committees,
                MAX(t.amount_tier) AS amount_tier
            FROM trades t JOIN members m ON m.bioguide_id = t.member_id
            WHERE t.issuer_ticker = ? AND t.tx_type = 'buy' AND t.pub_date >= ?
            GROUP BY t.member_id
            """,
            (ticker, cutoff),
        ).fetchall()

        sig["last_tx_date"] = sig.get("last_tx_date", "")
        sig["max_tier"] = sig.get("max_tier", 0)

        result = research_signal(sig, [dict(b) for b in buyers], dry_run=dry_run, force_deep=True)
        if result:
            researched.append((sig, result))

    conn.close()
    return researched

def research_all(min_score: float = SCORE_RESEARCH_THRESHOLD, dry_run: bool = False,
                 ticker_filter: str = None) -> list[dict]:
    """Research all signals above min_score. Returns list of (sig, result) pairs."""
    from analysis.scorer import score_tickers

    print("Re-scoring signals...")
    signals = score_tickers()

    # Apply filters
    eligible = [s for s in signals if s["total_score"] >= min_score]
    if ticker_filter:
        eligible = [s for s in eligible if s["ticker"].upper() == ticker_filter.upper()]

    print(f"Researching {len(eligible)} signals (score ≥ {min_score})...\n")

    results = []
    for sig in eligible:
        # Fetch full buyer details from DB
        conn = sqlite3.connect(DB_PATH)
        conn.row_factory = sqlite3.Row
        cutoff = (datetime.now() - timedelta(days=LOOKBACK_DAYS)).strftime("%Y-%m-%d")
        buyers = conn.execute(
            """
            SELECT DISTINCT
                t.member_id,
                m.first_name || ' ' || m.last_name AS member_name,
                m.party, m.chamber, m.committees,
                MAX(t.amount_tier) AS amount_tier
            FROM trades t
            JOIN members m ON m.bioguide_id = t.member_id
            WHERE t.issuer_ticker = ?
              AND t.tx_type = 'buy'
              AND t.pub_date >= ?
            GROUP BY t.member_id
            """,
            (sig["ticker"], cutoff),
        ).fetchall()
        conn.close()

        buyers_list = [dict(b) for b in buyers]
        # Attach for use in _build_prompt
        sig["last_tx_date"] = sig.get("last_tx_date", "")
        sig["max_tier"] = sig.get("max_tier", 0)

        result = research_signal(sig, buyers_list, dry_run=dry_run)
        if result:
            results.append((sig, result))

    return results

if __name__ == "__main__":
    parser = argparse.ArgumentParser(description="Research top congressional trade signals via LLM")
    parser.add_argument("--dry-run", action="store_true", help="print prompts without calling LLM")
    parser.add_argument("--ticker", help="research a specific ticker only")
    parser.add_argument("--min-score", type=float, default=SCORE_RESEARCH_THRESHOLD)
    args = parser.parse_args()

    results = research_all(
        min_score=args.min_score,
        dry_run=args.dry_run,
        ticker_filter=args.ticker,
    )

    if not args.dry_run and results:
        print(f"\n{'='*60}")
        print(f"Research complete: {len(results)} signals researched")
        print(f"{'='*60}")
        for sig, res in results:
            print(f"\n{sig['ticker']} (score={sig['total_score']}) — {res.get('conviction','?')} conviction")
            print(f"  Company:   {res.get('company_summary','')[:120]}")
            print(f"  News:      {res.get('recent_news','')[:120]}")
            print(f"  Catalyst:  {res.get('catalyst_hypothesis','')[:120]}")
            print(f"  Timeliness:{res.get('timeliness','')}")


Now let me check the backlog and CLAUDE.md to see if there are any open design decisions.


# CongressWatch — Backlog

Deferred concerns and ideas we've identified but don't have a clean solution for yet,
or that are lower priority than current work. Review periodically.

---

## Scoring

### Committee-sector mapping is too coarse
**Filed:** 2026-04-22

The `_COMMITTEE_SECTOR_MAP` in `scorer.py` is hand-coded and maps committee names to broad
sector strings. This creates two problems:

1. **Over-broad matches**: "Armed Services" maps to both `industrials` and
   `information-technology`. This means nearly any tech stock bought by an Armed Services
   member scores 3.0 for committee relevance, even if there's no plausible defense nexus.
   A member on Armed Services buying NVDA (plausible — AI chips, JWCC contract) and the same
   member buying a generic payroll SaaS company both get committee_score = 3.

2. **No company-level specificity**: The mapping is ticker-blind. A score of 3 means
   "the member's committee has broad jurisdiction over this sector" — it can't distinguish
   between a member buying the single company their committee directly regulates vs. any
   random name in that sector.

**What a good solution looks like**: Per-trade committee relevance assessment, likely via LLM.
At research time, ask the model to score (0–3) how specifically this member's committee
assignments relate to *this company's* primary business. Store as `committee_score_llm`
alongside the current heuristic score. Use the LLM score when available, fall back to
heuristic otherwise.

**Why deferred**: Adds an LLM call per signal at scoring time (before the research step),
increasing latency and cost. Also requires careful prompt design to avoid the model being
too generous. Better to accumulate more signals first and validate that the heuristic
misfires often enough to justify the extra call.

---

## Data Quality

### Win rate is measured against current market prices, not exit prices
**Filed:** 2026-04-22

`tools/win_rate.py` computes win rate as "fraction of positions that are currently
profitable." This is a snapshot, not a realized return:

- A position could be up today but will eventually be sold at a loss.
- A position that was down for months and recently recovered looks like a win.
- Bull market conditions inflate all win rates (most of the current data is from 2025–2026).

**Better approach**: For each member's trade, fetch price at `tx_date + 30d`, `tx_date + 60d`,
and `tx_date + 90d` as fixed-horizon returns. Win = positive return at some fixed horizon
(e.g., 60 days). This requires yfinance calls per trade (expensive for 14k trades) but could
be done as a one-time backfill and then maintained incrementally.

**Why deferred**: Requires significant yfinance API calls (one per trade × 3 horizons = ~40k
requests). Rate limiting and batching need design. Also the current win rate is still useful
as a directional signal; the flaw mainly affects members with small trade counts.

### Signal return uses first buyer's tx_date, not each member's individual date
**Filed:** 2026-04-22

`return_from_tx_pct` in the signals table is computed from `first_tx_date` — the earliest
trade date among all buyers within the current scoring window. If member A bought in January
and member B bought in March, the signal's return reflects January's price. Member B's actual
return would be different.

**Impact**: Moderate. For cluster signals where multiple members bought at similar times, the
error is small. For long-running signals with buyers spread across weeks, it can be
misleading (especially for volatile stocks).

**Fix**: Store per-member returns in the trades table directly. Requires a `price_at_tx` column
on the trades table and yfinance lookups per trade. Could be done lazily (only for trades that
become part of a high-score signal).

---

## Features

### Sell signals
**Filed:** 2026-04-22

If a member who previously bought a ticker in our window later discloses a sell, that's
potentially an exit signal. Currently `sell_count` is tracked in the scorer but not acted on.

**Implementation sketch**: In the notify step, check if `sell_count >= member_count` for any
currently held signal. If so, fire a separate ntfy notification on the `congressWatchSignals`
topic with a different title prefix (e.g., "EXIT SIGNAL: $TICKER").

### Options call vs put distinction
**Filed:** 2026-04-22

The `options_bonus` (+2) applies to any options trade regardless of direction. A congressional
put option is a very different signal from a call — it implies a member expects the stock to
fall. Capitol Trades sometimes includes the direction in `tx_type_extended`. We should parse
this and differentiate: calls get +2, puts might warrant a separate alert rather than boosting
the buy signal score.

### Sector clustering across members
**Filed:** 2026-04-22 (from original project_plan.md)

If 5+ members across different parties buy the same *sector* in the same week — regardless of
individual ticker — that may signal a macro view worth flagging. Not captured by current
per-ticker signal scoring.


Now let me check the CLAUDE.md to see the codebase documentation.


# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Common Commands - look typing here

```bash
# Setup (Python 3.9+)
python3 -m venv .venv && .venv/bin/pip install -r requirements.txt
cp .env.example .env  # add ANTHROPIC_API_KEY and/or GOOGLE_API_KEY

# First run on a new machine: load historical data, then initialize DB schema
.venv/bin/python db/migrate.py
.venv/bin/python tools/backfill.py

# Daily pipeline
python3 run_daily.py                # full run
python3 run_daily.py --no-ingest    # re-score + research + notify (skip scraping)
python3 run_daily.py --dry-run      # score + report only, no LLM, no notifications
python3 run_daily.py --force        # re-notify signals already sent today

# Individual steps
python3 analysis/scorer.py          # re-score and regenerate report (fast, no network)
python3 tools/price_fetcher.py      # refresh prices and market caps via yfinance
python3 ingest/committees.py        # refresh member committee assignments
python3 tools/generate_report.py    # regenerate markdown + HTML reports only

# Tests
.venv/bin/python -m unittest discover

# Interactive viewer (Flask, port 5001)
python3 viewer/app.py
```

The focused test suite currently covers notification policy decisions.

## Architecture

### Data flow

```
Capitol Trades (HTML scrape)
        ↓
  ingest/capitol_trades.py  →  members + trades tables
        ↓
  ingest/committees.py      →  members.committees (JSON, weekly cache)
        ↓
  analysis/scorer.py        →  signals table (rebuilt each run)
        ↓
  tools/price_fetcher.py    →  signals.price_* + return_* columns
        ↓
  analysis/researcher.py    →  research_cache table (persistent across runs)
        ↓
  analysis/notification_policy.py → high-signal notification gate + cooldown
        ↓
  tools/generate_report.py  →  reports/signals_latest.md + index.html (project root)
        ↓
  notify/ntfy_client.py     →  push notification via ntfy.sh
```

### Database (`data/congress_trades.db`)

Four tables. `members` and `trades` are append-only source data. `signals` is **rebuilt from scratch on every scoring run** — don't store anything there that needs to persist across runs. `research_cache` is keyed by ticker and survives across runs; the researcher checks `cached_at` and `deep_researched_at` before calling the LLM.

When adding a new column to any table, add it to both `db/schema.sql` and as an `_add_column_if_missing` call in `db/migrate.py`. Never run a bare `ALTER TABLE` in application code.

### Two-tier LLM research

`analysis/researcher.py` does two rounds per qualifying signal:
- **Basic research** (≥ `SCORE_RESEARCH_THRESHOLD`): company summary, recent news, catalyst hypothesis, timeliness assessment
- **Deep research** (≥ `SCORE_NOTIFY_THRESHOLD`, top `DEEP_RESEARCH_TOP_N` only): adds `risk_factors` and `recommended_action`; skips if the same ticker was deep-researched within `DEEP_RESEARCH_DECAY_DAYS`

`analysis/llm_client.py` is the provider abstraction — swap between Anthropic (`claude-sonnet-4-6`) and Google (`gemini-2.0-flash`) via `LLM_PROVIDER` in `config.py`.

### Scoring

`analysis/scorer.py` computes six score components (see README for the formula). Key subtlety: **timeliness multipliers are applied only at notification-decision time** in `run_daily.py`, not stored in the `signals` table — so the stored `total_score` always reflects the raw signal strength, making historical comparisons valid.

`analysis/notification_policy.py` is the push-alert gate. It requires deep research, high conviction, explicit buy/add action, limited sell pressure, enough independent edge factors, and a rolling same-ticker cooldown backed by `signal_outcomes`.

### Reports

`tools/generate_report.py` produces two outputs every run:
- `reports/signals_latest.md` + dated `reports/signals_YYYY-MM-DD.md`
- `index.html` at the **project root** (financial-news-style dark HTML page served statically)

The Flask app at `viewer/app.py` (port 5001) is a live dynamic view of the same DB — it is separate from the generated HTML and runs independently.

### Python 3.9 compatibility

All files that use `X | Y` union type syntax in annotations must have `from __future__ import annotations` as the first import. This is already present in all current files that need it; add it when creating new files that use union types.

## Configuration

All tunable values are in `config.py` (overridable via `.env`):

| Key | Default | Purpose |
|---|---|---|
| `SCORE_RESEARCH_THRESHOLD` | 5.0 | Minimum score to trigger LLM research |
| `SCORE_NOTIFY_THRESHOLD` | 9.0 | Minimum score before notification quality gates |
| `DEEP_RESEARCH_TOP_N` | 10 | How many signals get deep research per run |
| `DEEP_RESEARCH_DECAY_DAYS` | 7 | Days before re-researching the same ticker |
| `MAX_NOTIFICATIONS_PER_RUN` | 1 | Maximum push alerts per pipeline run |
| `NOTIFY_TICKER_COOLDOWN_DAYS` | 14 | Same-ticker rolling notification cooldown |
| `NOTIFY_MIN_CONVICTION` | `"HIGH"` | Minimum LLM conviction for push alerts |
| `NOTIFY_MIN_EDGE_FACTORS` | 3 | Independent edge factors required before push |
| `LOOKBACK_DAYS` | 30 | Trade history window for scoring |
| `LLM_PROVIDER` | `"anthropic"` | `"anthropic"` or `"google"` |


Perfect! Now let me check for any test fixtures and the predictor.py to understand if there's a "shadow mode" for prediction.


curl -s "https://raw.githubusercontent.com/jamesjlopez/congressional-trading/main/analysis/predictor.py" | head -800