14 local AI setups on a real app

Correct two-file patch. Repeated planning and test launches delayed completion; first write took nearly 35 minutes.

Runtime: LM Studio / llama.cpp

Settings: 118,016 effective context · one slot · native MTP, max 3 draft tokens

Independent combined suite: 92/92 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Fast but missed U+2011; inserted U+200B removal and repeated ASCII hyphen. Wrote tests but did not run them.

Runtime: LM Studio / llama.cpp

Settings: 118,016 effective context · one slot · native MTP, max 3 draft tokens

Independent combined suite: 78/90 passed. Fixed acceptance assertion failures: 8. Debug build: Pass.

Eventually correct after a long regex repair loop. 68 tool calls and 88.7 minutes; also expanded support to U+2010 and wrote scratch files outside its worktree.

Runtime: LM Studio / llama.cpp

Settings: 118,016 effective context · one slot · native MTP, max 3 draft tokens

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Verified native xhigh did not repair correctness. Recognized U+2011 but failed normalization; did not run its useful regression test.

Runtime: LM Studio / llama.cpp

Settings: 118,016 effective context · one slot · native MTP, max 3 draft tokens

Independent combined suite: 69/88 passed. Fixed acceptance assertion failures: 12. Debug build: Pass.

Diagnosed the intended fix but never applied it. Two Metal out-of-memory retries, unsupported structured requests, and long non-tool responses. Only wrote a scratch harness.

Runtime: mlx_vlm.server

Settings: 262,144 loaded context · MTP max 3 · KV 8-bit / group 64, starts at token 5,000

Independent combined suite: 71/87 passed. Fixed acceptance assertion failures: 16. Debug build: Pass.

Its own 92 tests passed, but production code and tests shared the same wrong character: U+200C instead of U+2011. Independent checks caught it. Two Xcode diagnostic timeouts also increased wall time.

Runtime: mlx_vlm.server

Settings: 118,016 Hermes context / 262,144 model context · extracted native MTP max 3 · KV 8-bit / group 64

Independent combined suite: 86/94 passed. Fixed acceptance assertion failures: 8. Debug build: Pass.

Retained usage and session evidence show 16 model calls without a usable patch or final response. The later status audit confirmed a clean benchmark worktree. DFlash was not exercised. Full independent Xcode results were not recovered.

Runtime: MLX / Hermes

Settings: Base MLX path; no DFlash drafter. Exact effective context and rendered reasoning tier not verified in retained summary.

Independent combined suite: Not recovered. Fixed acceptance assertion failures: Not recovered. Debug build: Not recorded.

Passed with 24 tool calls, no exact repeated calls, and no recorded context compaction. 48.3% less wall time than the historical Q6 control. About 12.88 GiB observed swap and seven memory-guard cache sheds remained. DFlash draft acceptance: 56.50%. This end-to-end speed difference is not an isolated on/off measurement of speculation.

Runtime: mlx-dspark / DFlash2

Settings: 118,016 context · DreamFoundries/Qwen3.8-27B-6bit + incoai/Qwen3.8-27B-DFlash2 · medium

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Correct two-line parser change and four authored tests. Recovered from an obsolete Xcode path and a faulty URL assertion; final independent suite passed. Test coverage has minor gaps described in the run report. Previously fastest passing local configuration; Signal later completed this task in 15m56s.

Runtime: LM Studio / llama.cpp 2.37.0

Settings: 118,016 effective context · one slot · native MTP, max 3 draft tokens

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Earlier passing local observation: 15m56s, 19.3% less time and 47.7% fewer output tokens than GSQ-RCO. More tool calls (36 vs 22), so speed did not mean fewer actions. Four authored tests; recovered from an invalid Xcode selector and two URL-encoding assertion failures. Explicit Unicode inspection and exact Markdown links were strengths. Used file search instead of graph; wrote a /tmp scratch file despite the scope constraint; edited production before tests. Independent 91/91 and build passed without repair. Different endpoint wrapper, q8 KV and MTP max 4 mean this is a complete-configuration comparison, not an isolated fine-tune effect.

Runtime: Standalone llama.cpp 2.37.0 / Hermes

Settings: 118,016 context · q8_0 K/V · native MTP max 4 · backend draft sampling off · vision projector loaded · temperature .6 / top-p .95 / top-k 20 / min-p .05 / presence penalty 1.0 · Xcode 27 beta 6

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Fastest passing local observation: 11m21s, 28.7% less time than Signal IQ3_S and 42.5% less than GSQ-RCO. 24 tool calls versus 36 and 22 respectively. Simpler two-line production patch; four authored tests with all-dash full Markdown links and direct raw-variant URL round-trips. Recovered from four URL-encoding assertion failures; corrected own 89-test suite passed, then independent 91/91 and build passed without repair. File search instead of graph, production-before-test ordering, /tmp diagnostics and piped exit-code handling remain workflow limitations. Unrelated TypeScript check lacked tsc and was disclosed. One run per configuration; cache, machine state and work trajectory can affect timing.

Runtime: Standalone llama.cpp 2.37.0 / Hermes

Settings: 118,016 context · q8_0 K/V · native MTP max 4 · backend draft sampling off · BF16 vision projector loaded · temperature .6 / top-p .95 / top-k 20 / min-p .05 / presence penalty 1.0 · same harness/runtime settings as Signal IQ3_S

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Passed in 17m03s: 91 independent tests and Debug build, without external patch repair. Two production lines and four authored tests. 16 tools and 5,619 output tokens, fewer than Signal Q6, but 50.2% more elapsed time. Used file search instead of graph and production-before-test ordering; recovered unaided from stale Xcode path and test-without-building before a build. Authored Markdown and URL checks are narrower than Signal Q6; fixed independent acceptance tests passed. Runtime, KV representation, no MTP and released Xcode differ from Signal. Single observation, not an isolated model-quality comparison. Vision passed a basic image smoke test only.

Runtime: mlx-vlm 0.6.15 / Hermes

Settings: 118,016 context · MLX 8-bit KV group 64 from token 0 · prompt cache on and observed · no MTP drafter · vision included and smoke-tested · temperature .6 / top-p .95 / top-k 20 / min-p .05 / presence penalty 1.0 (64-token window) · verified sampler correction · Xcode 27.0 released

Independent combined suite: 91/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

Incomplete: stopped by the controller after 60m 15s with no final agent response. Independent evaluation of the unmodified patch passed both fixed acceptance methods (eight dash/spacing cases) and the Debug build. The combined suite passed 90 of 91 tests; the sole failure was a model-authored Markdown test incorrectly expecting colon percent-encoding (%3A). The one-hour cutoff was chosen around minute 48, not preregistered or uniformly applied to earlier runs. Elapsed time at termination is not a successful completion time.

Runtime: LM Studio bundled llama.cpp 2.37.0, commit 8172e65; standalone server

Settings: Hermes 3aee290899e478c5fdfb6a241ef62758a49829b3; 118,016 context; q8_0 K/V; native MTP max 4; backend draft sampling disabled; temperature 1, top-p 0.95, top-k 20, min-p 0, presence penalty 0. Released Xcode 27; iPhone 17 / iOS 26.5. Same historical baseline, prompt and fixed evaluator. Vision projector loaded and image smoke test passed; coding benchmark was text-only

Independent combined suite: 90/91 passed. Fixed acceptance assertion failures: 0. Debug build: Pass.

The user-provided historical comparison lists Ornith 1.5 35B-A3B as failed. Detailed benchmark logs, timing, quantization, context, and test counts were not recovered. This is a reported result, not an independently revalidated measurement. It is not conflated with a separately found Ornith 9B download.

Runtime: Not recovered

Settings: Configuration named in the supplied screenshot; detailed runtime settings unavailable

Independent combined suite: Not recovered. Fixed acceptance assertion failures: Not recovered. Debug build: Not recorded.