# Root merge 4/5: Hermes Plugin Hackathon evidence All **25 inputs, 158 assessments, 98 entrants and two calibration references** are merged (counts derived from supplied plugin arrays). This is one root's evidence, not the final order: round-4 rankings/score.py done were not supplied, so five winners/five runners-up cannot be determined here. ## Sources and interpretation Vectors = Useful,Reliable,Polish,Creative,Beyond=total/rank; total U+R+P+C+2B, out of30. All158 totals recompute correctly. Ranks are batch-local; r0 marks references. All late standout flags=false. Zero vectors on inaccessible pins mean unscored, outside the1–5 scale. Tests: p/f/e/s/xf=passed/failed/errors/skips/expected failures; never pool different runs. [PLAN.md](https://gist.githubusercontent.com/gtmcknight/594c47336546c8763e641b46cf9af4ec/raw/PLAN.md) was read: field1,464 plus two references;147 first-read seats, top240 reread by24 seats, top120 tested by60 seats (three each), final20 ranked by12. Late/extra repair hung/missing reads. Family curves, reference calibration, empirical shrinkage and half-score/half Bradley–Terry deep standing require global results/score.py. Raw totals here cannot replace that process. No orchestration or fresh plugin testing is claimed by this merge. [Dxx]/[Xx]/[Lxx] link original supplied result paths and source all associated scores, observations, code judgments and caveats. PR numbers identify `https://github.com/NousResearch/hermes-agent/pull/{number}`. Separately mentioned sources.json, logs and failed-source identities were not supplied; underlying quotations cannot be individually audited. Pins are those reported by seats, never mutable HEAD substitutes. **†** marks requester-reported failed source rechecks: D49(1), D51(3), D52(1), D53(1), D55(1), D57(1), L11(6), L12(1), L21(1), L22(5), X1(1), X3(1). Discount externally sourced claims; local runtime reports remain attributed observations, not independently reproduced facts. Winner-critical flagged claims require pinned-code confirmation; inaccessible pages/author benchmarks/universal safety claims are not rehabilitated here. Code mapping: D44–48=b14/deep-h; D49–54=b15/deep-i; D55–60=b16/deep-j; L11/L12=b17/read1-late-001/002; L21/L22=b18/read1-late2-001/002; X1–4=b19/deep-x-01–04. [D44]: ../../../batches/deep-h/items/deep-44/result.json [D45]: ../../../batches/deep-h/items/deep-45/result.json [D46]: ../../../batches/deep-h/items/deep-46/result.json [D47]: ../../../batches/deep-h/items/deep-47/result.json [D48]: ../../../batches/deep-h/items/deep-48/result.json [D49]: ../../../batches/deep-i/items/deep-49/result.json [D50]: ../../../batches/deep-i/items/deep-50/result.json [D51]: ../../../batches/deep-i/items/deep-51/result.json [D52]: ../../../batches/deep-i/items/deep-52/result.json [D53]: ../../../batches/deep-i/items/deep-53/result.json [D54]: ../../../batches/deep-i/items/deep-54/result.json [D55]: ../../../batches/deep-j/items/deep-55/result.json [D56]: ../../../batches/deep-j/items/deep-56/result.json [D57]: ../../../batches/deep-j/items/deep-57/result.json [D58]: ../../../batches/deep-j/items/deep-58/result.json [D59]: ../../../batches/deep-j/items/deep-59/result.json [D60]: ../../../batches/deep-j/items/deep-60/result.json [L11]: ../../../batches/read1-late/items/read1-late-001/result.json [L12]: ../../../batches/read1-late/items/read1-late-002/result.json [L21]: ../../../batches/read1-late2/items/read1-late2-001/result.json [L22]: ../../../batches/read1-late2/items/read1-late2-002/result.json [X1]: ../../../batches/deep-x/items/deep-x-01/result.json [X2]: ../../../batches/deep-x/items/deep-x-02/result.json [X3]: ../../../batches/deep-x/items/deep-x-03/result.json [X4]: ../../../batches/deep-x/items/deep-x-04/result.json ## Shared execution limits Deep seats fetched field-pinned source, installed locked Hermes/Python3.14 (often3.14.10), used tryit.py/Nous FakeLLMServer/AIAgent and empty then planned/error calls. Completed fake-model plumbing does not prove model quality, live accounts/hardware or Desktop. D57 stalled before reports; D58 bootstrap exceeded30s executor limit: both rely on isolated capability probes/source/tests, not completed chats. Deferred schemas/empty added_by_plugin do not imply absent tools; memory/backend providers need selection. Hermes bases: D44/47/49/52/53/55/56=fdde27e7dae174cb4ba842589e92508852b7dd9f; D48=f0066531cefb84421abe9f5e645cbcd7afa63848; D51=b89e04eb7021a549db13d910b7a29483ead4f043; D54=5fac5ed7b5918185006761d85b28050e580c4ef8; D60=8ec227311bbbe7863d3982ce8f35012d6a59e7d7; X1/2=a84025ade22d2a4085f575d4353072e5b702635d (X3:a84025a); others report shallow current source. Version/layout differences matter. All deep catalog gates reportedly passed except D57 USSD capability loading; install/doctor/manual-review exceptions remain below. Reported harness-only fixes: loopback proxy bypass, Git boundaries/package-name copies, multiprocessing fork for denied AF_UNIX, preinstalled dependencies, disabled schema deferral, explicit provider/config and synthetic services; no entrant repairs. X1/2 discarded shared-home/proxy baselines; X2 forced exit after saving/closing agent. D47 separated admission/runtime profiles; D52/53/54/60/X1/2 sometimes bypassed repeated dependency publication through local config. Manual load success does not repair standard enable failures. Host failures are retained without automatically blaming code; green suites do not refute extra reproduced defects. Except individually stated tests, live providers/devices/Desktop remain unverified. No input proves AI-judge steering (X3 reports none in its set); contradictory Capalis JUDGES.md claims alone do not trigger reliability1. Strong local finishes are retained as ranks, not overall placements. ## Entrant evidence ledger For the ten inaccessible late entries below, descriptions are PR-only claims, not verified implementation. Both assigned seats retained all-zero vectors/standout=false; these mean unscored/missing, not valid 1–5 judgments or last places. Both report failed fetches and inaccessible pinned source/README/API; repository404/auth failures do not prove deletion, absence or broken functionality. No manifest/tests inspected; late source flags apply. ### agent-si · PR 149242 [D52]† `5,4,4,4,5=27/r2`. Selected memory provider remembered an exact Tuesday quote, searched/retrieved it with source/date and refused fabricated Friday evidence (4 calls). Local, dependency-free, cited/disputed facts improve memory; keyword retrieval misses synonyms, owner attribution needs a principal and top-level modules can collide. Private memory-schema import warning. 19 tests passed. Cross-agent history and Desktop were fixture/source assessments only. ### agent-webmcp · PR 153993 [L12]† `0,0,0,0,0=0`; [L21]† `0,0,0,0,0=0`. Documentation-capability discovery/mounting/assistant CLI/Chromium; repository system1970/hermes-agent-webmcp; pin ec1cc2727896c3230507ba7969bc4328f95899ca. Both report repository/API/pinned README404 and fetch/auth failure. ### agentline · PR 152976 [D51]† `4,5,3,4,5=26/r6`. Nine tools and a phone/SMS gateway loaded, but key gating hid tools before setup. Local REST/WebSocket stand-ins then proved status/transcripts, 911 refusal, unattended call/SMS denial, allowlisted routing/replies, late-answer SMS and reconnect; an approval-disabled fixture completed a mocked call. 38 unit tests and 26/26 Hermes-loop integration assertions passed. No real/billed call; paid US/Canada service and modest visible polish limit reach. ### annotary · PR 153900 [D54] `4,2,5,4,5=25/r3`; [X1]† `5,4,5,4,5=28/r2`. Both real pinned-v0.5.0-server workflows shared/read Markdown versions, anchored comments and refused bad names. D54:331p/3s; X1:330p/3s/1 process-inspection deselection; each Desktop29p (X1 needed SDK loader). No phone/review/wake loop. DISAGREEMENT: X1 reliability4 notes private gateway import/consent setup; D54 reliability2 finds accounts.token_for profile-token files and server._secrets secrets.json bypass Hermes store (rule11). Keep specific credential objection despite passing admission/functionality; X1 omission is not rebuttal, and X1 is source-flagged. ### argus · PR 152481 [D44] `4,4,5,5,5=28/r3`; [D52]† `4,4,5,4,5=27/r3`. 5 capture hooks/auxiliary/intent-skill review/person-approved application. D44:3 tool outcomes/1 turn. D52:cost backfill/missing-proposal refusal;stock fake text failed assignment,shaped responses created/judged1 intent with4 calls/0f. Both115p(D52 needed mypy). No live skill change/model judgment/UI. Private skill/core imports,hundreds of possible auxiliary calls/history only widens are costs. Creativity5/4 differs,not capture function. ### ariadne · PR 150596 [D49]† `4,5,5,5,5=29/r2`. Captured1 turn/3 model calls/2 tools;CLI-origin replay reported surface/tool drift. Separate TUI roundtrip:2 turns/4 identical model calls/3 identical tools/1 local auxiliary/0 leaks/0 hook violations/no real replay tool execution.21 unit p;E2E assertions passed. Strong offline regression capability. Streaming/dispatcher hooks excluded;UI/fork/scenario matrix/cross-version upgrade-bisect unrun. Ignored middleware manifest field did not stop registration. ### augur · PR 154055 [X1]† `4,5,5,4,5=28/r4`; [X2] `4,4,5,4,4=25/r6`. Both seats parked/listed item 1, marked needs_you, rejected missing IDs and generated a real weekly PNG. Persistent queue, bounded nudges, Desktop chip and guarded Friday research/drafting are substantive. X1: 101 tests p after a 100-pass/1-fail run missed a 0.3s bound by 0.000043s; X2: 101 passed (12 classifier tests included). X2 enable timed out at 120s dependency publication; explicit activation worked. Reliability/beyond differ 5/5 versus 4/4. No real classifier accuracy, Telegram buttons, Desktop or scheduled Friday execution established; auxiliary classification adds cost/latency. ### barehand · PR 153871 [D52]† `3,3,5,5,5=26/r6`. QMP/VNC computer-use provider reaches firmware/bootloader screens with snapshots, screen text, approval and flight recorder. Without QEMU, gating hid tools; synthetic QMP fixture in real Hermes showed running 128x96 display, capture, empty snapshot list and clean bad-input errors (5 calls). No guest, VNC hardware or OS repair verified. 724 passed/11 failed/7 skipped; 4 screen-text cases and RFB timeout passed focused rerun, stopped-VNC doctor still failed. Other failures include restricted sockets/process inspection and unconfigured auxiliary vision; suite remains non-clean. ### breadcrumbs · PR 153020 [D53]† `5,4,4,5,5=28/r1`. Selected memory provider and neutral demo: all 4 calls found green-tea turns, returned keyword zoom, exact 80 C/three-minute/two-minute source at level 1, and unknown-ID error. Exact-source recall/context-engine integration improves Hermes itself. 668 passed/25 skipped/1 timing-sensitive hive failure among 694; all 27 hive tests p isolated with direct loopback. Private island/log/seat/slot imports remain upgrade risks. ACE compression, IPFS/sharing, summarizer, Desktop and throughput claims were unverified live. ### capalis · PR 153221 [D47] `3,4,4,4,5=25/r5`; [D48] `4,3,4,4,5=25/r6`; [D59] `4,4,4,4,4=24/r2`. Approved/version-bound recipes, practice writes/read-back/dedup/drift quarantine are substantive; no live account write/phone approval. D47/48 status/setup/compile errors worked; D48 tokenless writer called runnable despite missing auth. D59 raw KeyError slack_channel; practice URL fetch lacks private-address block. D47:304p/1f/1s(closure WAITING,1 fail then2 passes); D48:303p/2f/1s(two replies WAITING); D59:305p/1s. Green rerun does not erase timing/notification issues. D59 contradictory JUDGES.md live-run claims are not proven judge steering. Score/rank disagreements preserved. ### ci-red-to-green · PR 153294 [D53]† `4,5,5,4,4=26/r5`. Hooks/commands/blueprint, not a new model tool: status/watch errors/demo/discard worked; real plugin-manager E2E reproduced 3 failing-before and 4 passing-after invocations, first-execution approval, push blocking and source-only patch proof against unchanged tests. All 96 tests passed. Useful code-enforced proof/review beyond suggesting a patch; constrained to pytest/go test on GitHub Actions and compatible local environment/model-generated fix. No live GitHub PR/account polling verified. ### clearvoice · PR 149978 [D53]† `3,2,4,4,5=23/r6`. Three accessibility tools loaded; speech queued spoken=false/awaiting_user=true, blank text, disabled file delivery and relative transcription path all refused cleanly (4 calls). Human single-use approval is substantive. 158p/4f/1s of 163: three reproducible failures from missing clearvoice_engine/data/piper_voices.json; fourth meets absent Phonon before relative-output assertion. Missing bundled voice index breaks non-English lookup. Speech tests mock engines; actual STT/TTS quality, weights, microphone and Desktop unverified. ### coach · PR 152596 [D49]† `4,4,4,3,5=25/r6`; [D50] `3,4,4,3,3=20/r5`; [D56] `4,4,4,3,4=23/r6`. 2 tools/7 hooks: no-model/empty/options/disabled-decision errors clean; bounded reviews/evidence/budgets are genuine hooks. D49/D56 scripted ctx.llm advice needed skip_same_model=false; default rejected fallback. No independent-opinion/paid Jev quality proof. D49:451p (449p/2f corrected fork/Git); D50:450p/1 lockfile permission failure; D56:448p/3s (446p/2f/3s corrected), UI check6 views/40 settings. Private pricing import/setup/inference cost remain. Beyond5/3/4 disagrees whether execution gate exceeds largely skill/subagent-replicable criticism. ### confidential-mode · PR 151301 [D44] `4,3,5,5,5=27/r5`. 11 hooks/middleware loaded; local-tier preflight rejected auto vision route, accepted local config and locked; lock blocked unattended web_search while allowing read_file. 3080p/3f/5e/8s/6xf: all 5 missing-home errors passed rerun; one failure absent temporary directory, two older built-in-tool expectations. Polished, creative privacy enforcement, but 31 private import sites, in-process isolation and shell/side-call escape limits make this a scoped lock, not universal confinement. ### construct · PR 150346 [D48] `3,4,5,5,5=27/r3`. 12 tools, approval hook and bundled engine/viewer built a hardcover project, checked 0 errors/warnings, emitted interactive 7-part artifact and animated 25,560-byte GLB read back as 7 parts. Bad project/format errors were initial plan mistakes. 39p/3f/6s plus 18 passed subtests across Hermes/NetLogo suites: thumbnail missing, film browser startup failed, cancellation test encountered already-failed films. No full engine suite, FreeCAD/STEP/NetLogo/film verification. Strong provenance-aware creation, specialized audience and browser/CAD dependencies. ### culprit · PR 152914 [D45] `4,4,4,5,4=25/r4`. Three tools/3 hooks snapshot file-changing calls into separate Git history; direct Engine baseline+3 steps bisected guilty step 2 and reverted it while keeping later edit. In Hermes, timeline errors were clear but two writes recorded 0 steps in fresh-home workspace; bisect/revert correctly awaited human approval. 29p/1s. Silent debug-only snapshot errors can lose the whole history; check_command accepts empty/dangerous shell commands, leaving approval as sole guard. Direct success is not successful chat capture. ### deck-diff · PR 153062 [L12]† `0,0,0,0,0=0`; [L22]† `0,0,0,0,0=0`. PPTX-versus-generated-lesson slide drift/integrity reports; Baophan00/deck-diff; pin 5c7f65460b2481da2055094408291bf6320ced1f. Both report repository/API/pinned README404 and fetch/auth failure. ### dress-rehearsal · PR 151804 [D44] `5,5,5,4,5=29/r1`. Two hooks and /rehearse loaded. Built-in calls, including missing-file error, returned; actual armed write was blocked/recorded without creating the file, show/off worked and malformed drop gave usage. 84 tests passed. Broad, polished pre-execution review with person-triggered replay earns this seat's first rank. Rehearsal is profile-wide; replay cannot adapt to effects of earlier recorded steps. ### excubitor · PR 150261 [X3]† `4,4,3,3,4=22/r6`; [X4] `4,4,4,4,4=24/r3`. Eight hooks and SQLite ledger really stopped repetitive terminal work: X3 says calls 1-6 ran, 7 sought approval, 8-9 blocked; X4 says 6th escalated at stagnant=6/threshold=6. Preserve this counting discrepancy, not one exact universal cutoff. Both 94p/2s. Receipts/spend ledger worked; gateway dedup, hang alerts and real spend caps unverified. X3: ledger errors fail open as zero spend, author duplicate demo hit Hermes idempotency, screenshot labeled mock. X4: status renders hard-stop budget 0 as $5/$2 via 'or' defaults; synchronous notifier can stall 8s. Headline caps/dedup are not runtime-proven by loop test. ### ferry · PR 150900 [L11]† `0,0,0,0,0=0`; [L22]† `0,0,0,0,0=0`. Asynchronous store-and-forward Hermes jobs via Git; teal-sea/ferry; pin 1dc7b99d7efbd53032f221063fdf1cb2823a22b6. Both report repository/API/pinned README404 and fetch/auth failure. ### fine-print · PR 153844 [D48] `5,5,5,5,5=30/r1`; [D49]† `5,5,5,4,5=29/r1`. Both observed50-result search warning/offset50 reaching model. D48:12,001-byte line truncation,ordinary missing-file error,/limits explanations/errors/7KB HTML report. D49:81 limits=16 verified+19 fixture-only+46 reference-only,no errors;0.0.0 package versus supported0.23.0 warning correct. Each487p/8s(reference-only/no-shape skipped). Broad zero-model/network-cost recovery guidance,not universal/version-independent detection. Both rank1;creativity5/4 yields30/29. Desktop delivery unrun. ### flowcanvas · PR 153240 [D58] `4,3,5,4,5=26/r5`. Catalog capability probe loaded workflow canvas but no completed Hermes-loop conversation (D58 bootstrap limit). Inspectable reusable model/tool workflows and polished Desktop need actual UI verification. 564p/2f/4e/2s of 572; failures/errors shown involved denied AF_UNIX multiprocessing. Raw window.hermesDesktop use triggered catalog warning. Large non-green suite and bridge warning remain; not proven broken solely by sandbox sockets. ### fluent · PR 150353 [D45] `5,5,5,4,4=27/r1`. Silent until /fluent adopt; adapted harness adopted ls/git/tar/rm. Real terminal hook blocked invented ls long flag against installed ls 9.4 help, allowed git --version, let prefix typo git log --onelin through for Git to reject, and routed git clean -fdx to approval (unattended: no execution). 442p/100s. Helpful version-specific guard and clear errors, but only long options, manual per-program adoption and deliberate prefix ambiguity limit protection. Three hooks, no new tool. ### gemini-live · PR 150251 [D50] `4,4,5,4,5=27/r3`; [D60] `5,4,5,4,5=28/r2`; [X1]† `5,4,5,4,5=28/r3`. Intentional0 tools/hooks; duplex Desktop/dashboard engine delegates/steers threads/recalls memory with human approvals. D50 load only; D60 CLI key/usage errors, status200/models missing-key/session412/unknown-tool error; X1 status200/session412/empty memory200/bad args400. Secret-scope keys/server-limited ephemeral tokens. Tests D50:126p/1 shell-path f,JS1p; D60:117p/10 path-policy f,JS39p; X1 subset21p/1 deselected, broad114p/10f/3 host deselections,JS39p. D60/X1 bundle checks passed. Hidden-ancestor/absolute/symlink guards reject legitimate paths. No live Google/token issuance/audio/mic/reconnect/delegation; JS uses scripted sockets. Score difference is reach/beyond, not successful voice call. ### genui · PR 152159 [D56] `4,5,5,4,5=28/r2`. Three tools/one Desktop hint hook showed task card, updated state, rejected bogus status with useful schema error and returned nonblocking next-turn ask_user instructions. All 53 tests p, including Chromium rendering/update/click-to-message through SDK/React stand-ins. Four predefined card types and persistent interaction layer improve Desktop tasks, but no Desktop launch; accurate card updates still depend on model directives. ### grimoire · PR 152691 [X1]† `5,5,5,4,5=29/r1`; [X2] `5,5,5,4,5=29/r1`; [X3]† `4,4,5,4,4=25/r3`. All seats guarded DuckDB attach/profile/grouping, missing-file/DELETE/DROP refusals and receipts. Different 4-row fixtures: X1 health100/100, North250/South200; X2 25% missing amounts, South250/North150, not inconsistent answers. Both held merge for approval and flagged invented999 in actual hooks (X1:q2; X2:2/3 numbers trace/needs revision). Custom selftest21 groups passed, not pytest (X3:about400 assertions). X3 scores25 versus29/29: regex footer is post-hoc, can annotate unrelated later numeric replies; digits/small coincidences do not prove semantic/text/ranking truth. SELECT-only/external/path lock,30s/row caps useful. DuckDB pyproject-only; X2 enable timeout/manual activation. Desktop/12 screenshots source-read; paid-model token/speed/scale claims unverified, not rebuttal of actual receipt check. ### hail-hermes · PR 154028 [D46] `3,3,5,4,4=23/r6`. Desktop panel, Wear OS voice/watch pairing and polished docs target personal devices; 4 device tools unavailable until paired (calls unknown-tool; malformed haptic step never ran). 366p/2 voice-backend failures/1s; character tests needing numpy not run, README claims 403. No paired device, watch/hub or real voice demonstrated. Private import warnings and gated usefulness temper polished presentation. ### hallway-test · PR 152475 [D45] `4,5,5,5,4=27/r2`. One tool plus /hallway implements 5 simulated accessibility testers with code-enforced low vision, keyboard/screen-reader, color and tremor constraints, browser replays/cards/Desktop. Real Hermes refused file:// and link-local metadata URL; demo gave actionable missing-browser error in isolated runtime. 35 tests p after Playwright/Chromium installation (before: 15p/19s/1f); root pytest import failed, tests/ invocation worked. Browser tests do not prove model-driven persona loop; many calls/minutes and browser setup constrain use. ### harness-story · PR 154078 [D45] `3,3,4,4,3=20/r6`; [D50] `3,3,3,3,3=18/r6`. Five tools/12 hooks reconstructed state.db sessions into offline causal stories/animated HTML/Desktop; list/generate(use_llm=false), inspect tally and missing-story errors worked. Both seats: 5p/1s (6 tests, skip needs real state.db), thin for about 4,000 lines. Unknown filesystem:write capability and author LAN narrative endpoint/private model defaults undermine install polish; external runtime declared. Scores differ 20/18 on polish/creativity, not offline functionality. ### heats · PR 152450 [D56] `4,4,5,4,5=27/r3`; [D60] `5,4,5,4,5=28/r1`. 3 tools/middleware/referee. D56 race awaiting approval,n99/unknown IDs refused;missing pytest error,unittest already-green baseline avoided race. D60 calculator edit:0p/1f,3 correction nudges then fake-model iteration limit;restored1p/0f,no nudge,completed. Each final211p after Git isolation(initial206p/5f). Ignored middleware manifest field did not prevent registration. Improves coding verification;no live multi-agent race/cancel/apply. Shell edits evade referee,dependencies shared with checkout,parallel inference costs. D60 higher usefulness has stronger red/green probe. ### helm · PR 150160 [X4] `3,3,5,4,4=23/r4`. Desktop-only control desk with shared approval/clarify queue, live plan, themes, avatars; empty register(), 0 agent tools/hooks, 3 read-only sprite routes. No Desktop session. Smoke: 31 contributions/13 rendered; API: 191 sprites, traversal refused. Rule-8 warning on about 27 data-slot theme CSS selectors, not DOM scripting. Finished docs/screenshots, some mocked; automatic avatars write assets despite click-only claim, New bot silently clones default profile with share_auth=true. UI changes do not establish new agent capability. ### hermes-ark · PR 149640 [X2] `5,5,5,3,5=28/r3`; [X3]† `5,4,5,4,4=26/r2`; [X4] `4,5,5,4,4=26/r1`. Manual checkpoints/human-only admin guards proved across seats. X3 blocked prune/delete, snapshot43 files/2.8MB, verified blobs; X4 blocked init/prune, sanitized label, list/diff memory, bad-id error. X2 checkpointed twice around memory with autosnapshots disabled; X4 saw no separate pre-memory snapshot. Automatic capture thus source/test-backed, not every runtime. Full suites174p/1 live-S3 skip each; valid package-copy required. X2 initial171p/3f/1s fixed proxy/timing; no-Hermes76p/96 integration skips. Useful encrypted/deduplicated item recovery, needs user init/storage/passphrase custody. X3:cryptography loaded despite empty python_dependencies, broad regex/fingerprinting defense. X2 enable timeout/manual load; no live S3/Desktop. Scores28/26/26 do not settle undeclared dependency or live automatic capture. ### hermes-canvas · PR 150644 [D46] `4,5,5,5,5=29/r1`; [X2] `5,5,5,4,5=29/r2`. 4 tools/7 hooks,plans follow writes/check exits. D46:2-box dependency plan,get/next ready delegate_task,bad-op refusal. X2:fresh15-byte write/check0 -> done,writes1/check_exit0;manual done refused for check-owned box. Each306p;X2 initial304p/2f then305p/1f fixed Git/timing. D46 private-import warning/X2 clean gate:host versions differ. No live Desktop/concurrent subagent;pattern-based attribution limited. Both29,different useful/creative split. ### hermes-colab · PR 150110 [D48] `3,4,5,4,5=26/r4`; [D53]† `4,4,5,4,5=27/r4`. 8 tools/supervisor/checkpoint recovery; empty status,doctor ready=false/exact OAuth setup,missing/private scripts,bad accelerator,unknown logs/sign-in errors clean. No Google/GPU/checkpoint/recovery/Desktop/delivery live. D48:61p/5s then5 Jupyter integration failures;kernel diagnostic ZeroMQ Operation not permitted. D53:61p/5s then3f before300s timeout;isolated handshake failed32s with kernel exit. Path/secret/lease/OAuth/sync/recovery unit paths passed. Host-restricted fixtures neither prove nor disprove real Colab protocol;3 versus4 usefulness reflects reach/setup. ### hermes-drawbridge · PR 152755 [D45] `4,3,4,4,5=25/r5`. One folder tool/2 approval hooks loaded: no shares listed, traversal without share refused with bare invalid-request code, request_access returned requestId and Desktop directive. Expiring chat-scoped shares/per-diff write approval extend remote Hermes to personal folders. Core live share/write needs Windows Desktop, absent. Only tests/smoke.mjs (Windows Node 24 suite) stalled on unsettled top-level await line 283 under Node 24.21; no Python suite. Thin 76-line README and code-only errors are material polish/reliability gaps. ### hermes-regress · PR 153146 [X1]† `4,1,3,4,5=22/r6`. Four capture hooks and /regress saved one case with 5 actual tool calls; missing case/file errors were clean. All 18 supplied tests p, yet extra code AND Hermes-loop probes retained a synthetic password unchanged in prompt, answer and write args (only results redacted). args_key discards content: same-path wrong-content write received exact fixture match; real write was blocked, sentinel preserved. Useful personal regression concept but reliability 1 for reproduced privacy and false-match defects. No complete cross-model regression/CLI preflight proved. ### hermes-roam · PR 150563 [X2] `4,4,4,3,5=25/r5`. Three tools/hooks made local encrypted sync generation 0 -> actual push generation 1 (dry-run 3 files), read-only pull preview and unauthorized apply refusal with user restore command. Empty bundle key misleadingly returned invalid-path. 604p/2f/1s, 6 blackhole Git timing tests excluded; 20k-path test 3.14s versus 2s passed rerun, >80-column status failure reproduced. Crypto/archive/merge/rollback/consent passed. Standard validation passed, enable exceeded 120s then fixture activation worked. No S3/R2/Git backend or second-device restore verified; portable agent state is valuable but remote robustness not established. ### hermes-ros · PR 149313 [D44] `3,4,5,4,5=26/r6`; [D55]† `3,5,4,4,5=26/r5`. Both used shipped mock, not real robot. D44: 0.1 m/s moved 0.01m; D55: default cap 0.5m/s, 0.2m/s for 1s moved 0.2m via 10 publishes then zero, camera 320x240 PNG; both blocked 3m/s, D55 e-stop latched until human /ros clear. Nine tools/safety hook. D44 70p/10 live skips; D55 70p/11s after PyYAML. Real ROS/rosbridge/ROS2/hardware unverified. Safety only covers ros_* paths, no actions/Nav2. Same total 26 despite reliable/polish split; specialist reach rather than lack of code novelty. ### hermes-security · PR 149611 [D58] `4,4,5,4,5=27/r3`. Catalog capability probe loaded local evidence-oriented security review with controlled validation and Desktop surface; no completed chat because D58 bootstrap limit. 363p/1f, process-writer blocked by AF_UNIX restriction. Model-led review and process-backed storage remain limits; successful static/probe assessment is not a live security-review demonstration. ### hermes-tape · PR 152782 [L12]† `0,0,0,0,0=0`; [L21]† `0,0,0,0,0=0`. Model/tool record-replay middleware; itsolaph/hermes-tape; pin 6bde9fc31b2c04f7a606f01bb44b9f1fe9b43733. Both report repository/API/pinned README404 and fetch/auth failure. ### hermes-teleoperator · PR 149685 [D47] `4,4,4,5,5=27/r2`; [D58] `4,4,5,5,5=28/r1`; [D59] `4,2,3,4,5=23/r5`. Conversation-scoped SSH native-tool receiver is substantial hardware reach. D47 enable source-path failure fixed via separate admission/runtime profiles;5 calls proved local status/switch, unknown-host/remote-search refusal, local remained. Receiver-v1.2.0 checksum matched:8p/1 chroot skip;UI34p; own153 tests/2f/5 error reports/16s(missing/proc ELF), no Rust build/live SSH. D58 only probe,target46p+40 subtests. D59 enable refused twice(editable distribution missing), calls unknown;153 tests/2f/6e/15s,no cargo. Reliability4/4/2 reflects admission versus standard enable, not actual SSH success. D47:remote path bypasses other pre_tool_call guards/native parity. D59:JSON-config SSH-key file reads versus secret store,dense Rust. Keep credential/readability objections despite catalog pass/private approval warnings; D58 'safely' is not general safety proof. ### hermes-time-machine · PR 149553 [D51]† `5,4,4,5,5=28/r2`. Five tools/5 hooks implement checkpoints/forks/rewind and memory of irreversible external effects across rewind/compaction. Overlapping data/workspace refused; separate workspace checkpoint/list/preview/effects worked, missing id refused, unattended restore denied. Actual protected-workspace write falsely marked irreversible/outside because classifier uses session cwd. 122 unittest tests p after >12-ancestor Git isolation (initial 121p/1 false enclosing-repo failure); pytest import unsuitable. No remote push/publish performed. Useful recovery with reproduced configured-workspace mismatch. ### hermes-tuneup · PR 152190 [D46] `4,4,5,3,4=24/r5`. Read-only per-profile/model compaction advisor with exact fixes and confirmation/backup for writes. Default/all/bad-profile calls worked; reported 96K trigger of 128K. Private unresolver adapter warning; 314p/6 adapter-contract failures after PyYAML (window/MoA expectations against current Hermes). Clear runtime diagnostics do not settle upgrade compatibility. ### hermes-ui-cards · PR 150556 [D44] `5,4,5,4,5=28/r2`; [D59] `4,4,5,4,4=25/r1`. 1 tool/3 hooks,17 card types/answer loop/text fallback;plan/compare(D44),plan/checklist(D59) rendered,invalid demo/one-option choice refused precisely. D44:463p/4f/12s(2 sockets,2 Python/JS parity),JS197p. D59:464p/3f/12s(2 cross-process permissions,1 real datetime-hour-24 parity under3.14),Node unrun. Not one green suite. Desktop separate/off by default,no live UI;empty demo-video placeholder.28/25 differs reach/beyond,not refutation of parity defect. ### hermes-ussd · PR 153766 [D57]† `4,2,4,5,5=25/r6`. Declared SMS/USSD gateway for feature phones, allowlist/outbox/rate limits is a creative reach extension. D57 capability probe FAILED importing aiohttp (declared runtime dependency absent); other manifest/loadability/security checks passed. No scripted calls, tests or Africa's Talking account path verified. This is missing dependency in that environment, not proof platform code itself is wrong; reliability 2 and deployment uncertainty retained. ### home-3d-workshop · PR 153024 [D45] `3,4,5,5,5=27/r3`. 14 tools/4 hooks simulator returned printer/AMS state, empty queue/upgrades and owner; missing queue start blocked, bad model link/OpenSCAD absence and schemas gave clear errors. 958p/1f/13s: CLI expects unknown but gets disabled. Substantive household-chat find/design/slice/queue/typed-owner-start/watch/photo workflow, zero-token watcher, secret-store key, Desktop and extensive docs. Real Bambu/Bambuddy printer and OpenSCAD designs unverified; specialized hardware reach and huge scope (730-line README) constrain day-one value. ### housemate · PR 152458 [D52]† `4,5,5,4,5=28/r1`. Seven tools/voice platform and Rooms UI with real loopback Hub/ControlServer/registry/timer/SQLite: listed Kitchen/Office, created/listed/cancelled 600s tea timer, rejected -1 duration (5 calls). 565p/1s/4 slow/model tests deselected; initial 25 failures from tools.py import shadow/proxy resolved via parent invocation/direct loopback. Tests include WebSocket pairing, approvals, restart, arbitration and Desktop contracts. Coherent reuse of spare browser devices, but no actual microphone/wake/STT/TTS playback/Desktop; gateway, paired browsers and downloaded wake/VAD models still required. ### hypnos · PR 150800 [D49]† `4,4,5,5,5=28/r3`; [X2] `3,4,5,5,5=27/r4`. Capture/habits/personal-LoRA promotion/routing changes Hermes. D49:1 turn/2 tools captured,empty habits/export/usage errors,trainer/Docker missing guidance;sleep nothing_to_learn. X2:preference saved,1 turn/no brain,1 habit/10 probes,invalid diff exit2,passthrough completed. Both229p/2s(real trainer/Docker),X2 Hermes/isolated reruns. Fake nights are not learned weights;needs stack/data/sustained use. GPU/training/promotion quality/drift/timelapse unverified;router only Chat Completions/Anthropic Messages(D49). Ignored manifest field did not stop middleware. Usefulness4/3 differs. ### joust · PR 151376 [D51]† `4,4,5,4,5=27/r3`; [D56] `4,3,4,4,5=25/r5`. 8 tools/4 hooks/worktree middleware ran predetermined3-contender demo,hash-bucket-index won,status persisted,bad/missing args refused. Proves Git/testing/ranking,not live subagent repair. D51:94p/2f/1s of97;D56:93p/3f/1s+3 subtests,nonrepo fixed via Git boundary;UI55 checks/2 warnings(extra fetch,malformed directive stuck). Shell hex portability and symlink-clamp failure remain;extra probe shows pre_tool_call veto blocks returned escape path. Copy fallback recursed under enclosing repo;shell not sandboxed. Crossfire/approved merge useful,not full isolation;reliability/polish4/5 versus3/4. ### kumi · PR 152258 [D51]† `3,4,5,5,5=27/r4`; [D58] `4,4,5,5,5=28/r2`. 30 tools/8 hooks, DAW/audio/listening/history/sample-map UI and approvals give deep music workflow, niche runtime/DAW setup. D51 status needs pinned runtime 1.24.81, setup plan gives digest without install, connect setup_needed, read unavailable pre-setup. No live DAW/audio. D51 492p/2s/1f/1e of 496 (home-abbreviated path expectation; fixture path >1024). D58 targeted 96p +238 subtests/1f from top-level tools.py shadowing spawned approval import; no chat. Private approval import warning remains. Different targeted scopes/counts are not contradictory full-suite claims. ### lethe · PR 153302 [D55]† `5,5,5,4,5=29/r1`. Private-chat mode: before activation memory write allowed; actual /lethe then vetoed memory add, skill creation and malformed memory call before persistence (3 receipts), usage/status counted vetoes; /lethe-end deleted transcript and independent state.db query found 0 session rows. 67 tests passed. Scoped persistence control/deletion is a useful real agent change; external memory per-turn sync remains outside protection, not universal erasure. Background review callback-tested only; live external provider and gateway/Desktop exit unverified. ### lineage · PR 149244 [X1]† `4,3,5,4,5=26/r5`. 10 observer hooks passively linked real CSV read/model call to written report; initial verify fresh, changed North 100->900 stale. But deleting source left summary fresh, and rereading changed CSV made unchanged old report fresh again: shared source hash overwritten, artifact edge lacks historical hash. Extra actual-capture defects survive all 19 Python/2 Desktop tests passing (initial missing PyYAML corrected). Differentiates observed/evidenced/inferred provenance but unreliable freshness claim; no remote refetch/Desktop visualization demonstrated. ### local-hands · PR 153443 [D49]† `5,3,5,4,5=27/r5`; [D50] `4,4,5,4,5=27/r2`; [D51]† `5,3,4,4,5=26/r5`. Pinned-TLS outbound backend/local approvals extend native terminal/files. All seats registered but caught AF_UNIX Operation-not-permitted on status/use; no pairing/remote approval/execution proved. D49 pytest22p/37f/19e; unittest59 reported/38e. D50:59/38e(21p reported). D51:22 successful methods/37 method errors+class setup error;19 Exec methods unrun. Counting differs; none proves78-test pass. Hyphenated-root import/socket failures are layout/host limits. D50:default0.0.0.0 bind; D51:active machine shared across chats, private EnvironmentConnectionError import. Generic errors/setup attack surface remain; scores do not certify live safety. ### local-scout · PR 153872 [D48] `5,2,4,4,5=25/r5`. Normal enable repeatedly failed editable source path despite admission/initial empty run(reliability2). Manual consent/config then masked email/card in prompt/read_file;all6 captured provider requests lacked originals. Savings net$7/break-even125;report counts only;dataset-consent/negative-n refused. 282p/1f/4s+160 subtests;failure optional own_cnn PLAN_ONLY/missing torch,logreg/knn/embed trained. Package-copy fixes not source repair. No broad accuracy/auxiliary masking/learned-savings guarantee from workaround/synthetic example. ### look · PR 152918 [D55]† `3,3,4,5,5=25/r6`. 14 tools/7 hooks/2 middleware; calibrated hover but chrome_connected=false, read said no active task, invalid choice rejected, benchmark returned synthetic resolution only (4 calls repeated direct schemas). Distinctive point-and-ask/local speech/browser approvals; Apple Silicon source beta/native setup restricts reach. 365p/16f +393 subtests after numpy/huggingface dependencies: 14 AF_UNIX permission failures, setup/private-directory assertions remain. Private native-input/benchmark imports and publication retry warning (later validation passed). No live gaze accuracy, supported Chrome, microphone, pointing or Desktop tested. ### lumina · PR 153365 [D57]† `5,4,5,5,5=29/r2`. D57 capability probe loaded 10 Photos/privacy tools, hooks and skill; polished on-device leak-prevention product/docs/fixtures, macOS-only. tryit never completed; no Photos/Vision hardware path verified. Test collection hit missing fastapi, no final count. Score 29/rank 2 is a source/probe judgment under substantial runtime uncertainty, not a demonstrated product run. ### muster · PR 151448 [D54] `4,2,4,4,5=24/r6`. Two tools/14 hooks durable worker fleet ledger, real lifecycle handles and privacy-aware health/control surfaces. Normal enable twice failed missing editable muster source; manual local activation made 4 calls return empty fleet, empty-goal refusal, one fanout handle and RUNNING/has_handle=true. 34 tests passed. No completion/Stop/Message/live Desktop proved; manual success does not repair standard install, reliability 2 retained. ### neural-mesh · PR 149750 [D54] `4,2,4,4,5=24/r5`. Selected fully local memory provider remembered/recalled dated deployment fact and refused bad date (5 calls); ordinary unrelated recall no match but dated unrelated query wrongly returned deployment fact (extra probe). Catalog passed; runtime doctor FAILED obsolete on_pre_compress/on_memory_write and declared/registered mismatch. 23 tests p, not coverage of reproduced relevance-floor defect. Hook-contract break and dated false positives justify reliability 2 despite working basic memory. ### no-cap · PR 149272 [D50] `4,4,5,4,4=25/r4`; [D52]† `5,2,5,4,5=26/r4`. D50 only terminal hooks; D52 actual failing pytest plus scripted All tests pass produced CAP citing1 failed/exit1;125/125 synthetic labels matched; real conflict doctor exited1. Not general accuracy/blind-eval proof. D50:5378p/1f/1s/17xf(doctor); D52:5375p/3f/2s/17xf(2 enclosing-Git artifacts,mocked doctor;real probe succeeded). 17xf=14 intentional abstentions+3 redaction defects; all3 --runxfail FAILED, credential fragments persisted after output windowing. Reliability4 versus2 reflects stronger privacy evidence, not broken primary calls. Green catalog/thousands of tests do not rebut leak. Heuristic misses/misflags remain;UI unverified. ### nvidia-switchyard · PR 152256 [D54] `4,2,4,4,5=24/r4`. 5 hooks/request middleware/provider/CLI/presets/UI. Status missing key,VGR list,bad-tier error;enable>120s then genuine retry passed. 174/174p,0s,1115.07s,documented bundled layout/real nemo-switchyard0.4.0a1/MockTiers;earlier layout failures superseded,external traffic test-guarded. No live model/subscription pair. Rule11 concern:providers._runtime obtains raw api_key/credential_headers makes Codex bearer instead of credentialed_provider_request/requires_auth. Green routing tests/admission do not rebut reliability2 credential objection. ### on-air · PR 153960 [D58] `3,4,5,5,5=27/r4`. D58 isolated capability probe only; live-show platform with guarded cohosting, Twitch reads, OBS outputs and approval desk is unusually complete but specialized. 19 targeted manifest/runtime-adapter tests p; broader selection timed out, no completed Hermes chat, live stream/OBS/Desktop observed. Do not equate reliability 4/source polish with demonstrated live-show operation. ### openinvest-hosted · PR 151482 [L11]† `0,0,0,0,0=0`; [L21]† `0,0,0,0,0=0`. Portable skills for hosted investment ledger with separately configured OAuth MCP; longsizhuo/openinvest-site; pin f1f8f850b264d5d55371f551e1823e205010eaf8. Both report repository/API/pinned README404 and fetch/auth failure. ### own-compute · PR 154010 [D54] `4,3,5,4,5=26/r1`; [D60] `4,3,5,4,5=26/r4`; [X3]† `4,4,5,4,5=27/r1`. Runtime cloud-compute:15 tools/2 hooks; status/host types/list/unknown-instance errors and X3 empty Slurm jobs worked. No actual Docker/SSH/Slurm/EC2/agent/billing. D54:3174p/13f/30s; rerun2p/11f(9 Stop/collect not_fetched,timer,path). D60:3172p/15f/30s; rerun2p/13f(repo/stub paths,timer,notification; HTTP readiness/display fixed). X3:3177p/10f/30s, attributes sandbox-stop/sudo to host. Preserve diagnoses, not green/all product bugs. X3 adds billed-EC2 approval gap: only pre_tool_call gate, no handler check, approvals-off permits no-card start (README admits; author live EC2 unrun). Secret scope/readable6,700-line UI does not rebut it. Reliability3/3/4 unresolved; X3 source-flagged. ### permission-canvas · PR 151341 [D48] `4,4,5,5,5=28/r2`; [D59] `3,3,4,5,4=23/r6`. Backend element read/act/hide/exact-once hash approval and private preview are substantive, limited to this backend/trusted policy/grounded accessibility geometry. No OS capture/input: unavailable headless/no cua-driver; D48 explicit native config still unavailable, ControlBusy first-run handoff, empty/unknown approvals clean. D48:314p/9s synthetic loader/protocol; D59:312p/2f/9s, subprocess venv lacked Hermes deps. Windows/Tk validation, dense one-line code/long docs and untested native UI qualify score28 versus23, not observed5-point functional loss. ### plur1bus · PR 151604 [X4] `3,2,3,3,4=19/r6`. Memory-provider concept (LanceDB/embedding/reranker, optional mood/dreaming/Obsidian) is substantive but enable FAILED editable plur1bus-hermes distribution missing. Doctor imported 0 tools/hooks, memory status installed but built-in stayed active; unavailable without ML dependencies. No catalog-folder tests; repository's reported 874 tests not run because stack could not install. Private memory-schema import, plaintext config apiKey acceptance and 26k-line ported OpenClaw readability remain objections; no memory_store/recall exercised. ### pr-ready-checker · PR 152198 [L12]† `0,0,0,0,0=0`; [L21]† `0,0,0,0,0=0`. Eleven-check pre-PR gate/Git pre-push hook; phon-t/pr-ready; pin f7733dd79da977c1d10e29565a3cad07408e7619. Both report repository/API/pinned README404 and fetch/auth failure. ### procgen-studio · PR 150293 [D57]† `5,4,5,5,5=29/r1`; [D59] `3,3,5,4,4=23/r4`. Image/video provider writes reproducible editable SVG/Python with source/seed, sandbox rendering/critic loop, gallery/Desktop. D57 probe passed but tryit stalled, no final tests. D59 enabled dependencies but fresh Hermes had no selected image backend; image_generate unavailable, no render verified. Importable-copy suite 77p/98f/3s from missing system libcairo; in-place pytest 9 had 9 collection errors. Cairo/ffmpeg undeclared system needs; bad /procgen render --size abc uncaught ValueError. Score 29/rank1 probe judgment versus 23/rank4 stronger setup/error evidence; preserve both, favor concrete caveats over untested product promise. ### project-command-center · PR 152899 [D47] `4,4,5,3,5=26/r3`. Three tools/2 hooks persisted active project card, PR link and board; bad lane schema-refused. Shared durable session/issue/PR state and polished Desktop target GitHub developers. Eight Python script suites passed (445 printed checks after Git isolation). Node 50p/6f, all six require full-history Hermes at hard-coded home path; fixtures passed. No live GitHub/Desktop. Readable implementation, but non-self-contained graph fixtures and niche UI reach constrain retention. ### promise-keeper · PR 151929 [D47] `5,4,5,4,5=28/r1`. User-confirmed persistent rules/self-protecting execution middleware. Five calls allowed note write/read,missing-file error,blocked rules.db overwrite;prod rule proposed,not armed. Real E2E19/19p:discovery/confirmation/write-delete-patch/middleware-only guards/revoke. 664p/11f/8s/2xf after Git isolation(4 earlier artifacts fixed);11 remaining Windows-form failures on Linux,Windows unverified. No UI click test. Shell best-effort,opaque tools/later rewrites remain gaps. Seat rank1 supported,not universal enforcement. ### retouch · PR 153090 [D55]† `4,5,5,4,5=28/r2`. Four image tools; human-style mark seeded via actual Workspace on synthetic 128x96 PNG. get/edit exposure +1/reply(needs_review, no aesthetic claim)/export/invalid path all worked; version 2 changed 813 inside pixels, 0 outside, compared, export existed. 266 unit/integration +6 property/state/concurrency tests p (80 warnings). Mark-specific agent threads, exact compositing/versioning form a concrete new workflow. Desktop required to make marks; no real generative provider/aesthetic review/UI launch, removal needs configured provider. Initial harness project-id plumbing corrected, not plugin defect. ### rich-paste · PR 152408 [D57]† `4,4,5,4,5=27/r5`. Rich HTML/Markdown conversion with image positions, request middleware/Desktop solves a real paste limitation. D57 capability probe passed (private benchmark-import warning), but no completed chat; interrupted pytest showed many failures/skips with no final count. Strong source-read polish cannot establish working paste/attachment delivery; failures remain unquantified, not silently treated as passing. ### roomsmith · PR 153724 [D53]† `3,5,5,5,5=28/r2`; [D55]† `4,5,5,4,5=28/r3`. Shared revisioned SQLite geometry/undo/budget/GLB/UI. D53 fixture4x5x2.5m,desk1.2x0.75x0.6m,$1,000/rev2,3 fits;D55 different4x3x2.5m/$800,711-byte export. Both width-1 refused without mutation,not inconsistent states. D53:255p;D55:254p/1s/2 warnings;each tsc/13Node p. Both28 with different useful/creative. No live Desktop/photo/merchant extraction;real measurements/human review needed,retailer data may be absent. Strong episodic niche product. ### salts-secrets · PR 151411 [D57]† `5,4,5,5,5=29/r3`; [D60] `4,4,5,5,5=28/r3`. Owner/public profile gates and approved notes enable personal-agent sharing. D57 declared/public registration mismatch,14 gateway-import collection errors,no chat. D60 explains role-conditional hooks:owner tool/2 hooks;3 calls safe note pending /salt approve,empty/password refusal;public3 calls empty lookup,terminal/file:// blocked. 374p/3f/1s:isolation correctly flags inherited Git context. ssh_dir_access is screening rule,not demonstrated theft. Both reliability4;29/28 usefulness spread. In-process multiplex gateway needs careful isolation;live Telegram/Discord routing,approval/publication/revoke unverified. ### samskara · PR 149823 [D60] `4,4,4,4,5=26/r5`. Selected memory provider with embedded demo returned 3 entities/15 provenance facts, queued 1 triple, refused DELETE, healthy status/no conflicts (5 calls). 150p after direct loopback (initial 147p/3 proxy-distorted failures). Local relational recall/conflict journal improves memory with offline demo and scoped secrets. Writes require manual reindex; embedded backend narrow read-only query shapes; session extraction skipped without ctx.llm. No external graph/extraction model exercised. ### sentinel-vision · PR 153596 [D56] `4,3,4,5,5=26/r4`. Nine tools/watchers/vision slots, change gate, bounded model budget/event memory/feedback. Basic empty history/rules, bad source/missing video/setup errors worked; synthetic folder recipe created/listed but vision_calls_per_hour=0 paused judging. 1678p/1f/2s: snapshot temp file disappeared at replace; failure passed alone, but deterministic prune-between-flush-and-replace reproduced StoreError by deleting active .tmp. Retain actual race, not dismiss as flaky test. No real visual accuracy/camera/screen or outbound alert delivery proved; in-process gateway coupling and model/device setup remain. ### shadow · PR 153955 [D46] `4,4,5,5,5=28/r2`. Six tools/12 hooks/3 middleware load; setup check clearly refused/no save, read/trial-step guarded outside trial (3 calls, no crash). Blind both-order candidate evaluation on personal conversations, undoable model switching and docs/tests are creative core improvements. 1887p/1 adapter-isolation failure/6s; acceptance/ not run. 40+ private imports/~19k lines, live-provider/verify setup and absent real model quality test make upgrade/setup fragility material, unverified successful trial. ### sheets · PR 152421 [X4] `4,5,5,4,4=26/r2`. Four tools plus grids skill/Desktop spreadsheet: initial no-access error, created typed grid from list/object rows, read data with injection disclaimer, bad-row proposal sent nothing and listed issue, unknown grid error named existing grids. 14 Python +161 Node tests passed. Human approval/undo/call-count previews make controlled shared data editing useful; main application/UI needs Desktop open, headless users see proposal inbox. No live Desktop rendered claim. ### signet · PR 153144 [D46] `4,4,4,4,5=26/r3`. Two mail tools/3 hooks loaded. No from_address and hostile address/header/attachment drafts refused, outbox worked; no mail sent. Exact-message SHA-256 approval holds even YOLO/scheduled runs. 107p/1 timing failure of 108 (2.5s vs 2.0s, likely load). Private imports, narrow email transport setup (mailer/SMTP/address) and only refusal paths demonstrated; approval design not live delivery proof. ### skill-court · PR 150873 [D57]† `4,4,5,5,5=28/r4`. Capability probe registered skill lifecycle/tool/LLM hooks, middleware/auxiliary work/Desktop: experiments quarantine harmful skills and support evidence-backed appeals, a novel self-improvement loop. D57 tryit stalled and tests interrupted by environment/network wrapper; no final count. Private appeal import and background/gateway coupling leave upgrade/automation uncertain, unverified measurement outcomes. ### smart-dictation · PR 151371 [X3]† `3,4,3,3,5=23/r5`. Local Whisper transcription provider/auxiliary correction, 0 tools/hooks; empty chat load only. Code returns coded errors for unselected/uncached/missing/oversize/busy and never auto-downloads weights. 85p/3 PCM-duration failures (av absent in test interpreter despite validator dependencies resolving); Node 87p/6s/0f of 93. Reported CI spans 3 OS and bundle rebuild. Needs manual provider/cached model; opaque errors, dense caveat README/one screenshot, no actual transcription/Desktop demo. ### solana-token-facts · PR 150658 [L12]† `3,3,4,3,3=19`; [L21]† `3,3,4,3,3=19`. Both fresh reads agree 19, no standout: eight read-only token/developer-history tools, /token facts and local Markdown/optional watch, authority/metadata/holders/funding/launch analysis, source links, scoped RPC secrets, bounded retries/offline fixtures. Both identify same correctness defect: unknown/failed graduation lookups counted as not graduated, turning missing evidence into a negative fact despite scan-limit disclosures. No money-moving claim; niche useful service/tool access, not core agent transformation. ### summon · PR 153861 [D54] `3,4,5,4,5=26/r2`. Real X platform with no tools; actual adapter requirements/empty config false, no-credential connect false with setup guidance, private notice success without public post. 32/32 unittest passed (allowlist, budget, privacy, signing/splitting/video/mock relay). Wrangler relay E2E aborted uv_interface_addresses Unknown system error 1, so no relay connection proof. Potential user-owned Hermes thread/skill learning workflow; paid X/account or limited author relay dependence, no live polling/post/learning run. ### syrinx · PR 153938 [D47] `4,3,5,5,5=27/r4`; [D55]† `5,3,5,4,5=27/r4`. Both night policies queued with /late;disabled in fixture for flow/park/status/errors/end. D47:30min block,git push held;D55:15min,bad minutes,flow0 ended parked1. Each64p;no timer wait/macOS/UI. Static probes:requests.post/execute_code,browser_exec POST,scp/rsync unheld. 'Nothing leaves machine' exceeds regex guarantee despite git hold;reliability3/private pet import both. Swapped useful/creative give27/27. D47 ranks PCC above despite lower total because safety overclaim. ### takeover · PR 151964 [D46] `4,3,4,4,5=25/r4`; [D53]† `5,3,5,4,5=27/r3`. One request tool/browser fencing solves human auth/2FA interruptions across UI. D46 valid request gave explicit missing TigerVNC/Xfce package guidance; actual handoff unavailable. D53 simulated readiness/screen, disabled bootstrap: wait0 posted, blank reason refused, wait1 timed out cleanly (3 calls), not live control. D46 reports '116 run:82p/34f/6e' (internally ambiguous total; preserve as reported); D53 core71:62p/1f/8e, plus Desktop10p/WebUI36p. Many sockets/input/lease errors restricted AF_UNIX, wording test unmocked installer lookup. Private Bot Desktop/interrupt imports, in-process WebSocket and privileged first-use setup remain risks; no real sign-in/browser/Desktop demonstrated. Useful/polish differ, reliability3 agrees. ### telegram-ops · PR 150814 [L11]† `0,0,0,0,0=0`; [L22]† `0,0,0,0,0=0`. Telegram micro-cards/action gates/forum-topic tools; brukmars/hermes-telegram-ops; pin 88c8855dde4fd4bb5dfb7f6130720a48a8b172af. Both report repository/API/pinned README404 and fetch/auth failure. ### thoth · PR 150497 [D50] `4,4,5,5,5=28/r1`. 19 tools/3 hooks shared board/worktrees hand tasks to real Claude Code/Codex CLIs, cross-review/merge on user's logins. No brain status gave clear thoth_init guidance; missing assign/plan args schema-refused. 77p/6s. Full delegation/review/merge not run without CLI/login/Git setup; broad tool surface and specialized coding workflow qualify strong core-extension idea, not proven completed autonomous team. ### thread · PR 152418 [L11]† `0,0,0,0,0=0`; [L22]† `0,0,0,0,0=0`. Append-only session closure ledger and next-session resume-context injection; pin 88ce440aa063447ee53fa4b369e63397322028c0. Both seats could not fetch/read the exact pin; L11 reports not-our-ref/API422/README404, L22 No commit found for SHA. L22 reports then-current main 4d9687e; this is not an authorized substitute or proof why history changed. ### tickover · PR 151968 [D60] `3,3,5,5,5=26/r6`. 10 tools/3 hooks local acoustic feedback: fleet/setup and bounded input errors; real encoder/ffmpeg synthetic selftest all 4 healthy/drift/start/no-change checks true, faulty fan localized 3.9-8kHz. 172p/5f/1s initially; dev deps fixed 2 PyYAML failures, 3 remain: ffmpeg9.0.2 crafted-playlist baseline, RTSP relay timeout, repairer notify watch setup. No live microphone/machinery/alerts. Narrow maintenance audience and unresolved stream/security/alert regressions; working local synthetic analysis is not physical fault accuracy proof. ### triage · PR 154048 [D47] `4,2,4,4,5=24/r6`; [D52]† `4,4,4,4,5=26/r5`. Two tools/4 hooks/middleware replace always-present skill index with retrieval/load and compaction-aware bookkeeping. Both exercised correct search/body/empty/unknown errors; D47 demo auto-attached then explicit load/dedup with auto_accept_threshold=0. Both139p/2 Windows skips. Distinct unresolved defects: D47 manual rule8 violation desktop/plugin.js:839-845 inline core jump-button styles and :1440 !important override despite automated pass; D52 fresh sync adopted pre-disabled git-commit then restore re-enabled it (unrelated private-skill stayed disabled). Shared new-chat state may cross windows. Reliability2 versus4 is substantive review difference; neither defect rebuts the other. Private imports/selectors warned; no Desktop launch. ### trust-ladder · PR 149738 [D51]† `5,5,4,4,5=28/r1`; [D56] `5,5,4,4,5=28/r1`. Both 5-call Hermes-loop plans learned bad/good-example guard, blocked forbidden terminal execution, created objective verifier task, ran verifier through Hermes terminal with passed=true and rejected invalid/empty action. Each final96p; D56 initial95p/1 timing assertion passed isolated/full rerun. Both28/rank1. Persistent correction enforcement/catch ledger/standing reminders improve real agent, but pattern matching can over-block or miss aliases/script indirection and fails open on hook errors; 'impossible' wording too strong. Formal/statechart coverage in tests is not model-checked proof. ### understory · PR 152879 [L11]† `0,0,0,0,0=0`; [L21]† `0,0,0,0,0=0`. Desktop music-label family tree/release provenance/SQLite forest; projectmehari/understory; pin f22c33c3d40d095bf6f1aea6241207c4b77c6e89. Both report repository/API/pinned README404 and fetch/auth failure. ### undo-send · PR 152063 [D59] `4,4,3,4,4=23/r3`. Tool-execution middleware actually held git push for grace window (28.2s conversation vs about3s), then released; Git failed128 outside repo as expected. Local echo unheld, pending.db created; bare CLI send_message absent, no messaging exercise. 64 tests passed. /unsend/Desktop chip/dashboard substantively add undo window to outbound actions, but regex coverage and no screenshots limit polish. README fail-open claim only wraps classifier: SQLite hold error would raise, not execute fallback. ### vipernet-model-manager · PR 150929 [L11]† `0,0,0,0,0=0`; [L22]† `0,0,0,0,0=0`. Per-turn/task provider-model routing, usage/decision ledger and dashboard; pin 6884fb77a7270bbf426ad562f755a5086ef933f7. Both seats could not fetch/read the exact pin; L11 reports not-our-ref/API422/README404, L22 No commit found for SHA. L22 reports then-current main faa855b; this is not an authorized substitute or proof why history changed. ### wake-me-when · PR 151135 [D44] `4,4,5,5,5=28/r4`. Seven deferred stream tools/hook/CLI: bundled keynote demo watch started/listed/stopped and rejected empty URL/condition. Standalone115p/7s; Hermes venv112p/3f/7s from plugin storage shared across tests. Concrete continuous-watch capability with polished docs; ffmpeg needed, external live services/media ingestion/message delivery unverified. Isolated green tests do not establish service connectivity. ### wearos · PR 151546 [X3]† `3,3,4,5,5=25/r4`. Three tools/hook/TTS/CLI: status actionable missing-tailnet/binary checks, pair opened 300s loopback window for human code, send refused no watch/missing text. 176p/1 resolver-name failure due checkout named wearos. Tailnet-bound human pairing is thoughtful, but load rewrites tts.provider to wearkit-gateway, starts listeners/runs CLI; raw 6-digit code stored beside hash, profile API_SERVER_KEY sent over plain tailnet HTTP, watch app absent from pin. No watch pairing completion/delivery/voice weights; substantial security/setup objections survive catalog sudo caution only. ### whos-listening · PR 153867 [D49]† `5,4,4,4,5=27/r4`. Selected provider (enable alone insufficient) exposed remember/recall/forget/review; private memory saved/recalled, invalid tier refused; later tool-search context held private save for outside-content review and withheld pending recall. /memories owner/config/errors worked. 524p final; initial523p/1 snapshot rewrite failure leaves flakiness caveat. Route-aware scrubbing/private-memory ownership useful, but compression/auxiliary calls may bypass guard and carry private memories. No remote sync/Venice duplicate judge/live provider-switch proof; not complete privacy boundary. ### workgraph · PR 150453 [D58] `3,2,5,5,5=25/r6`. Persistent branching/comparing/combining/replay product-state graph is creative/polished by source. D58 capability probe passed, no chat (bootstrap limit). Targeted tests7p/8f: interface got proxy502 instead of401, several candidate-flow failures also remained. Do not explain all away as proxy or treat unrun workflow as reliable; core versioning confidence low (reliability2). ### xray · PR 149414 [X4] `3,2,4,5,4=22/r5`. Two hooks/commands detonated demo sunny-weather RED with file:line bait-read/honeytoken-egress/clock-warp evidence, clean-calc GREEN; verdict list and missing-id errors worked. 13 tests passed. Fresh plugin-security idea, but --warps abc raised uncaught ValueError (reliability2); Python-level chamber can be disabled by tested plugin via sitecustomize globals/XRAY_EVENTS. No screenshots; successful demos are not adversarial containment proof. ## Calibration references (excluded from entrant/winner count) ### done-bell [D58] `3,5,4,2,3=20/r0`; [L11]† `4,3,3,2,4=20`; [L12]† `4,3,3,2,4=20`; [L21]† `4,3,2,2,4=19`; [L22]† `3,3,3,2,3=17`. Reference. Turn/approval hooks,atomic local state/history,chime/mute/busyBySession chip,no credentials/content transmission. D58 probe/no chat,9p;late reads cite9 stdlib/backend tests,not Desktop/OS delivery. All late reads:user-specific Windows path/manual pane copy. L22:2.5s polling,error-swallowing handlers,common idea. Scores20/20/20/19/17 reflect polish/reach/beyond,not conflicting operation. Late source flags apply. ### homeassistant [D58] `4,5,5,3,5=27/r0`; [L11]† `4,4,4,2,5=24`; [L12]† `4,4,4,3,5=25`; [L21]† `4,4,4,3,5=25`; [L22]† `4,5,4,3,5=26`. Reference. Official state_changed gateway/four REST tools/cron,scoped HASS_TOKEN,entity filters/cooldown/backoff/heartbeat. D58 probe only,target49p/1s,private import;no live device/account/chat. Late24/25/25/26 versusD58 27 is grading spread. DISAGREEMENT:L22 reliability5/'timeouts everywhere' versus L11/L12/L21 receive_json auth/subscription lacks per-message deadline(possible heartbeat-disabled hang). Do not repeat universal timeout guarantee. Domain-blocklist/optional filters and key-gated tools require deployment/token;no live safety proof. ## Coverage All25 inputs/158 score records are represented; none missing/empty. Ten inaccessible late entrants stay unscored; Solana-token-facts has two usable agreeing late reads. Both references are excluded as entrants. Passing checks are not formal catalog approval, and reviewer concerns are not official rejection. All roots and the prescribed final order are needed for five places/five runners-up.