# Marcus Aurelius local baseline — 2026-08-23

## Scope

Real public-source acquisition, normalization, split, local embeddings, local style
extraction, package build, and local fidelity harness. No persona roster expansion.

The original frontier-API baseline remains unavailable: the Anthropic key is invalid and
the valid OpenAI key has no API credits. These are local-model comparison results, not a
replacement for the frontier-judge acceptance gate.

## Thresholds

| Metric | Threshold | GPT-OSS 20B | Ornith 1.0 9B | Qwen 3.8 27B |
| --- | ---: | ---: | ---: | ---: |
| Groundedness | >= 0.70 | 1.00* | 0.80* | no result |
| Style match | >= 0.60 | 1.00* | 0.80* | no result |
| Misattribution | 0 | 0* | 0* | no result |
| Voice p50 first audio | < 2.0s | 6.303s (fail) | no complete run | no result |

\* The current local eval harness is not an independent fidelity test: it supplies the
holdout passage itself as the supposed generated answer and asks the same local model to
judge it. Treat these as package-pipeline smoke signals, not evidence of real groundedness
or style fidelity.

## Candidate results

### `openai/gpt-oss-20b` (GPT-OSS 20B MXFP4 GGUF)

- Downloaded locally (12.11 GB), loaded with llama.cpp, 16k context, parallelism 1.
- Clean code-block stream/stop probe.
- Full compile completed into `/private/tmp/mentorcore-gptoss-personas`.
- Eval record: groundedness 1.00; style match 1.00; misattribution 0; 5 samples.
- Ten-run voice benchmark: p50/p95 STT 1.491s/2.442s; retrieval 24ms/2.494s;
  LLM first sentence 524ms/1.372s; TTS first chunk 3.908s/15.744s; total first
  audio 6.303s/21.082s. The 2.0s first-audio gate fails.

### `ornith-1.0-9b`

- Loaded with llama.cpp, 16k context, parallelism 1.
- Clean code-block stream/stop probe.
- Full compile completed into the normal Marcus package.
- Eval record: groundedness 0.80; style match 0.80; misattribution 0; 5 samples.
- The first real voice turn reached audio in 23.930s, already failing the 2.0s gate.
  Its interrupted first benchmark is not reported as p50/p95.

### `qwen3.8-27b` (GGUF Q4_K_S)

- Loaded with llama.cpp, 16k context, parallelism 1.
- Clean code-block stream/stop probe.
- Timed out in the first local style-extraction call after 4:01; no package or eval
  result. This is a model-fit failure for the current compiler prompt, not a quality rank.

## Cost and failure modes

- API spend: $0 for all local runs. Local electricity/thermal cost was not metered.
- Wikipedia initially rejected the anonymous HTTP client (403); the compiler now sends a
  MentorCore User-Agent and acquisition succeeds.
- Anthropic fallback is unavailable because the configured key returns 401.
- OpenAI API authentication succeeds but the account has no API credits.
- The latency bench had hard-coded Anthropic construction, preventing local-provider
  measurement; it now uses the provider factory.
- STT and especially Kokoro TTS dominate current local voice latency; GPT-OSS's LLM p50
  itself is within the 700ms first-sentence budget.

## Verdict

The local package build gate passes for GPT-OSS and Ornith, but the stated real baseline
does **not** pass: neither candidate meets the <2.0s voice-latency requirement, and no
frontier-independent fidelity measurement exists yet. GPT-OSS 20B is the recommended local
compiler/runtime candidate for the next optimization iteration.