# Root merge 2 of 5: Hermes Agent Plugin Hackathon judging (25 results) This merge combines 25 seat results: round-2 reads `read2-018` to `read2-024` (batches read2-d and read2-e) and round-3 in-Hermes runs `deep-01` to `deep-18` (batches deep-a, deep-b and deep-c). All 25 input files were present, parsed as JSON and were non-empty. Between them they hold **192 seat scores of 125 distinct plugins**, plus 14 scores of the two reference plugins. The seat id in brackets after a claim (for example `[deep-10]`) is the result file that claim comes from. Every figure below is copied from those files. The only figures this merge computed are the means, ranges and seat counts, and how they were made is stated. ## How to read this - **Scale.** Each criterion is scored 1–5. Total = U + R + P + C + 2×B, out of 30. Every one of the 192 totals was rechecked against that formula, and none disagreed. - **Read2 seats** judged from the pinned code only. Each scored about 10 plugins plus the two reference plugins. Their entries carry a `standout` flag (★ in the table) and no rank. - **Deep seats** ran six plugins each inside real Hermes, through the catalog check (`hermes plugins validate`), enable, doctor and `tryit.py` against Nous's scripted fake model. Each deep seat then ranked its own six plugins from 1 to 6. A rank is that seat's judgement and does not always follow its totals: deep-01 ranked 3ds-platform (26) 6th, below four 25s, and deep-14 ranked hermes-ui-cards (27) above googlehome (28). - **Where the deep seats ran.** - Most built Hermes at `fdde27e7dae1…`: deep-02, 03, 04, 05, 07, 08, 09, 10, 13, 14, 15 and 17. - deep-06 used `77d34519…`, deep-12 used `20d111ea…` and deep-16 used HEAD `ff21b120`. - deep-01, 11 and 18 used a depth-1 clone of HEAD and did not record the commit. - Hermes defers plugin tools behind `tool_search`/`tool_call`. Every deep seat reports that the empty `added_by_plugin` list therefore does not mean a plugin has no tools. - No deep seat had real accounts, paid models or devices: no Desktop GUI, microphone, camera, watch, 3DS, GPU, AWS or X. Each seat's `setup` field says so. - **Mean** is the plain average of the raw seat totals. It is *not* `score.py`'s curved and reference-corrected figure. The PLAN says totals are curved per judge family (Claude/Codex), but these results do not name their seat's family, so the curve cannot be applied here. - **Judge-steering.** No seat reported a plugin that tries to steer an AI judge. deep-09 states that it found no judge-steering instruction. deep-11 notes that lethe's manifest carries "a paste-in prompt telling the agent to edit the plugin", but scored lethe Reliable 4 and did not treat that as judge-steering. ## Sources that did not hold on re-read (†) The requester reports that these results cite sources that failed when their machines read them again: | Result | Failed sources | |---|---| | read2-019 | 1 | | read2-021 | 1 | | read2-023 | 1 | | deep-03 | 2 | | deep-04 | 2 | | deep-06 | 2 | | deep-08 | 2 | | deep-10 | 1 | | deep-12 | 2 | | deep-14 | 3 | | deep-15 | 1 | The `sources.json` files these refer to are not among the inputs, so this merge **cannot tell which claim each failed source backed**. Every score from these seats is marked † below. Weigh their quoted-source claims accordingly; claims those seats made from their own runs (test counts, call outcomes) do not depend on a web source. The places where a † seat is the only evidence for a decisive point are named in the findings. A second, smaller note: the PLAN (BASE/PLAN.md, "Before the final") says that "seat read2-021 never answered". A full read2-021 result is present among these inputs, so it evidently landed later. ## Reference plugins: how hard each read2 seat graded The reference plugins show each read2 seat's own scale. On them, the seven read2 seats varied by only 3 points: done-bell 18–21 and homeassistant 25–27. read2-024 was slightly harsher on done-bell (Beyond 3) and slightly kinder on homeassistant. read2-023 was the kindest on done-bell (21). Reference findings that held across seats: - **done-bell:** real lifecycle hooks; the install guide uses an author-specific Windows path; tests cover Python state rather than the Desktop notification path. [read2-018] - **homeassistant:** a full gateway adapter; entity restrictions are opt-in, so safe device control depends on the operator. [read2-018] | Seat | done-bell U-R-P-C-B=T | homeassistant U-R-P-C-B=T | |---|---|---| | read2-018 | 4-3-2-2-4=19 | 4-4-4-3-5=25 | | read2-019† | 4-3-3-2-4=20 | 4-5-4-3-5=26 | | read2-020 | 4-3-3-1-4=19 | 4-5-4-2-5=25 | | read2-021† | 4-3-3-2-4=20 | 4-5-4-3-5=26 | | read2-022 | 4-3-2-2-4=19 | 4-4-4-3-5=25 | | read2-023† | 4-4-3-2-4=21 | 4-5-4-3-5=26 | | read2-024 | 4-3-3-2-3=18 | 5-5-4-3-5=27 | ## All 125 plugins: every seat's scores Sorted by mean raw total, then by number of seats. † = a seat with source caveats (see above); ★ = read2 standout; [n] = that deep seat's rank of its six. | # | Plugin | PR | Seats | Mean | Range | Each seat: U-R-P-C-B=Total [deep rank of 6] | |---|---|---|---|---|---|---| | 1 | argus | 152481 | 1 | 29.00 | 29 | deep-13 5-4-5-5-5=**29** [1] | | 2 | housemate | 152458 | 1 | 29.00 | 29 | deep-15† 5-5-5-4-5=**29** [1] | | 3 | breadcrumbs | 153020 | 2 | 28.50 | 28–29 | deep-14† 5-4-5-5-5=**29** [1]; deep-18 4-4-5-5-5=**28** [1] | | 4 | shadow | 153955 | 2 | 28.50 | 28–29 | read2-019† 5-4-5-4-5=**28** ★; deep-09 5-4-5-5-5=**29** [1] | | 5 | tenure | 153714 | 2 | 28.50 | 28–29 | read2-024 4-4-5-5-5=**28** ★; deep-03† 5-4-5-5-5=**29** [1] | | 6 | fluent | 150353 | 1 | 28.00 | 28 | deep-06† 5-4-5-4-5=**28** [1] | | 7 | grimoire | 152691 | 1 | 28.00 | 28 | read2-021† 5-4-5-4-5=**28** ★ | | 8 | kumi | 152258 | 1 | 28.00 | 28 | read2-020 4-4-5-5-5=**28** ★ | | 9 | lineage | 149244 | 1 | 28.00 | 28 | deep-06† 4-5-5-4-5=**28** [2] | | 10 | no-cap | 149272 | 1 | 28.00 | 28 | read2-022 5-4-5-4-5=**28** ★ | | 11 | retouch | 153090 | 1 | 28.00 | 28 | read2-021† 4-5-5-4-5=**28** ★ | | 12 | say-do-gate | 149365 | 1 | 28.00 | 28 | deep-08† 5-5-4-4-5=**28** [1] | | 13 | sentinel-vision | 153596 | 1 | 28.00 | 28 | deep-13 4-5-5-4-5=**28** [2] | | 14 | sheets | 152421 | 1 | 28.00 | 28 | read2-022 5-4-5-4-5=**28** ★ | | 15 | wake-me-when | 151135 | 1 | 28.00 | 28 | deep-13 4-4-5-5-5=**28** [4] | | 16 | workroom | 152651 | 3 | 27.67 | 26–29 | deep-02 4-5-5-5-5=**29** [1]; deep-03† 4-4-5-5-5=**28** [3]; deep-11 4-5-5-4-4=**26** [2] | | 17 | googlehome | 153826 | 2 | 27.50 | 27–28 | deep-13 4-4-5-4-5=**27** [5]; deep-14† 4-5-5-4-5=**28** [3] | | 18 | hermes-ui-cards | 150556 | 2 | 27.50 | 27–28 | read2-019† 5-4-5-4-5=**28** ★; deep-14† 5-4-5-3-5=**27** [2] | | 19 | tironian | 151281 | 3 | 27.33 | 26–28 | read2-019† 4-4-5-5-5=**28** ★; deep-04† 4-2-5-5-5=**26** [2]; deep-10† 4-4-5-5-5=**28** [1] | | 20 | twring | 153357 | 3 | 27.33 | 26–28 | read2-018 4-4-4-4-5=**26**; deep-13 5-4-5-4-5=**28** [3]; deep-15† 5-4-5-4-5=**28** [3] | | 21 | agent-si | 149242 | 3 | 27.00 | 27 | read2-023† 5-4-4-4-5=**27**; deep-03† 5-4-4-4-5=**27** [4]; deep-15† 5-4-4-4-5=**27** [4] | | 22 | vellum | 152533 | 3 | 27.00 | 26–28 | read2-018 4-4-5-5-5=**28** ★; deep-04† 3-4-4-5-5=**26** [4]; deep-05 4-3-5-5-5=**27** [2] | | 23 | hermes-canvas | 150644 | 2 | 27.00 | 26–28 | deep-09 5-4-5-4-5=**28** [2]; deep-18 4-5-5-4-4=**26** [2] | | 24 | pinpoint | 149426 | 2 | 27.00 | 27 | read2-021† 4-4-5-4-5=**27** ★; deep-14† 4-4-5-4-5=**27** [4] | | 25 | trust-ladder | 149738 | 2 | 27.00 | 26–28 | read2-024 4-4-5-5-4=**26** ★; deep-12† 5-5-4-4-5=**28** [1] | | 26 | cmi | 151887 | 1 | 27.00 | 27 | read2-019† 5-4-4-4-5=**27** | | 27 | excubitor | 150261 | 1 | 27.00 | 27 | read2-021† 5-4-4-4-5=**27** | | 28 | flowcanvas | 153240 | 1 | 27.00 | 27 | deep-09 5-4-5-3-5=**27** [3] | | 29 | helm | 150160 | 1 | 27.00 | 27 | read2-021† 5-4-5-3-5=**27** | | 30 | hermes-design-studio | 154067 | 1 | 27.00 | 27 | deep-08† 4-4-5-4-5=**27** [4] | | 31 | hermes-lens | 153316 | 1 | 27.00 | 27 | deep-11 3-4-5-5-5=**27** [1] | | 32 | home-3d-workshop | 153024 | 1 | 27.00 | 27 | deep-02 3-4-5-5-5=**27** [2] | | 33 | imprint | 153978 | 1 | 27.00 | 27 | deep-14† 4-4-5-4-5=**27** [5] | | 34 | kata | 150840 | 1 | 27.00 | 27 | deep-01 4-4-4-5-5=**27** [1] | | 35 | numogram | 151674 | 1 | 27.00 | 27 | read2-023† 3-4-5-5-5=**27** ★ | | 36 | nvidia-switchyard | 152256 | 1 | 27.00 | 27 | read2-022 5-4-4-4-5=**27** ★ | | 37 | project-command-center | 152899 | 1 | 27.00 | 27 | read2-018 5-4-5-3-5=**27** | | 38 | rich-paste | 152408 | 1 | 27.00 | 27 | deep-12† 5-4-5-3-5=**27** [2] | | 39 | undo-send | 152063 | 1 | 27.00 | 27 | deep-08† 5-4-4-4-5=**27** [3] | | 40 | wrenchboard | 153911 | 1 | 27.00 | 27 | read2-018 3-4-5-5-5=**27** ★ | | 41 | 3ds-platform | 149202 | 3 | 26.67 | 26–27 | read2-019† 3-4-5-5-5=**27**; deep-01 2-4-5-5-5=**26** [6]; deep-03† 3-4-5-5-5=**27** [5] | | 42 | contract-necromancer | 152908 | 4 | 26.50 | 25–28 | read2-022 5-4-5-4-5=**28**; deep-05 5-4-5-4-5=**28** [1]; deep-07 4-5-4-4-4=**25** [2]; deep-17 4-5-4-4-4=**25** [2] | | 43 | seedguard | 151543 | 4 | 26.50 | 25–28 | read2-020 5-4-4-4-5=**27**; deep-03† 5-4-5-4-5=**28** [2]; deep-06† 5-3-4-3-5=**25** [6]; deep-10† 5-3-4-4-5=**26** [2] | | 44 | confidential-mode | 151301 | 2 | 26.50 | 26–27 | deep-06† 3-4-5-4-5=**26** [5]; deep-07 3-4-5-5-5=**27** [1] | | 45 | foreman | 153851 | 2 | 26.50 | 25–28 | deep-08† 5-4-5-4-5=**28** [2]; deep-17 4-4-5-4-4=**25** [1] | | 46 | local-scout | 153872 | 2 | 26.50 | 26–27 | read2-023† 5-4-4-4-5=**27** ★; deep-15† 4-4-4-4-5=**26** [5] | | 47 | shadow-army | 153524 | 2 | 26.50 | 25–28 | read2-020 5-4-5-4-5=**28** ★; deep-12† 4-2-5-4-5=**25** [5] | | 48 | habits | 153606 | 2 | 26.00 | 24–28 | deep-06† 4-4-5-5-5=**28** [3]; deep-07 4-3-4-5-4=**24** [5] | | 49 | agentline | 152976 | 1 | 26.00 | 26 | read2-024 4-4-4-4-5=**26** ★ | | 50 | clotho | 152582 | 1 | 26.00 | 26 | read2-023† 4-3-4-5-5=**26** | | 51 | hermes-colab | 150110 | 1 | 26.00 | 26 | deep-13 3-4-5-4-5=**26** [6] | | 52 | hermes-maps | 152871 | 1 | 26.00 | 26 | read2-022 4-4-5-3-5=**26** | | 53 | hermes-teleoperator | 149685 | 1 | 26.00 | 26 | read2-022 4-3-4-5-5=**26** | | 54 | joust | 151376 | 1 | 26.00 | 26 | read2-020 4-4-4-4-5=**26** | | 55 | keryx | 153853 | 1 | 26.00 | 26 | deep-15† 4-4-5-5-4=**26** [6] | | 56 | learnloop | 153821 | 1 | 26.00 | 26 | read2-022 4-4-5-3-5=**26** | | 57 | own-compute | 154010 | 1 | 26.00 | 26 | deep-04† 4-3-5-4-5=**26** [3] | | 58 | permission-canvas | 151341 | 1 | 26.00 | 26 | deep-11 3-4-4-5-5=**26** [3] | | 59 | plur1bus | 151604 | 1 | 26.00 | 26 | read2-020 5-4-4-3-5=**26** | | 60 | roomsmith | 153724 | 1 | 26.00 | 26 | read2-019† 3-4-5-4-5=**26** | | 61 | salts-secrets | 151411 | 1 | 26.00 | 26 | deep-05 4-4-4-4-5=**26** [3] | | 62 | smart-dictation | 151371 | 1 | 26.00 | 26 | deep-12† 4-4-5-3-5=**26** [4] | | 63 | syrinx | 153938 | 1 | 26.00 | 26 | read2-020 4-3-5-4-5=**26** | | 64 | thumbs-fix | 153796 | 1 | 26.00 | 26 | read2-020 4-4-4-4-5=**26** | | 65 | flota | 153549 | 3 | 25.67 | 25–26 | deep-03† 4-2-5-5-5=**26** [6]; deep-04† 4-2-5-4-5=**25** [5]; deep-18 4-2-5-5-5=**26** [3] | | 66 | benchforge | 152504 | 2 | 25.50 | 25–26 | deep-01 3-4-5-5-4=**25** [3]; deep-10† 3-5-5-5-4=**26** [3] | | 67 | culprit | 152914 | 2 | 25.50 | 24–27 | read2-019† 5-4-4-4-5=**27**; deep-11 4-4-4-4-4=**24** [4] | | 68 | hail-hermes | 154028 | 2 | 25.50 | 25–26 | read2-023† 4-3-5-4-5=**26**; deep-01 3-3-5-4-5=**25** [5] | | 69 | hallway-test | 152475 | 2 | 25.50 | 25–26 | read2-024 4-4-5-5-4=**26** ★; deep-05 4-4-5-4-4=**25** [5] | | 70 | look | 152918 | 2 | 25.50 | 25–26 | deep-09 4-3-4-5-5=**26** [4]; deep-17 3-3-4-5-5=**25** [4] | | 71 | neural-mesh | 149750 | 2 | 25.50 | 25–26 | deep-02 4-3-4-4-5=**25** [4]; deep-12† 5-3-4-4-5=**26** [3] | | 72 | summon | 153861 | 2 | 25.50 | 25–26 | read2-019† 4-4-4-3-5=**25**; deep-08† 4-4-5-3-5=**26** [5] | | 73 | hermes-drawbridge | 152755 | 2 | 25.00 | 22–28 | deep-04† 5-5-4-4-5=**28** [1]; deep-07 3-4-3-4-4=**22** [6] | | 74 | lethe | 153302 | 2 | 25.00 | 24–26 | deep-05 5-2-5-4-5=**26** [4]; deep-11 3-4-5-4-4=**24** [5] | | 75 | omniscience | 154025 | 2 | 25.00 | 22–28 | deep-15† 5-4-5-4-5=**28** [2]; deep-16 4-2-5-3-4=**22** [6] | | 76 | atomicmail | 153491 | 1 | 25.00 | 25 | read2-019† 4-4-4-3-5=**25** | | 77 | check-please | 152323 | 1 | 25.00 | 25 | read2-018 5-3-3-4-5=**25** | | 78 | codecoach | 152207 | 1 | 25.00 | 25 | read2-020 4-4-5-4-4=**25** | | 79 | docket | 153876 | 1 | 25.00 | 25 | read2-021† 3-5-5-4-4=**25** | | 80 | hermes-notify | 149220 | 1 | 25.00 | 25 | read2-022 3-4-4-4-5=**25** | | 81 | inset | 153904 | 1 | 25.00 | 25 | read2-023† 4-4-5-4-4=**25** | | 82 | lights-out | 153822 | 1 | 25.00 | 25 | read2-024 4-4-5-4-4=**25** | | 83 | locus-codegraph | 153687 | 1 | 25.00 | 25 | deep-01 4-4-5-4-4=**25** [2] | | 84 | mandate | 153050 | 1 | 25.00 | 25 | deep-02 3-4-5-5-4=**25** [3] | | 85 | no-manual | 152309 | 1 | 25.00 | 25 | read2-024 3-4-5-5-4=**25** ★ | | 86 | phone-senses | 153363 | 1 | 25.00 | 25 | read2-023† 4-4-5-4-4=**25** | | 87 | privacy-xray | 150696 | 1 | 25.00 | 25 | deep-16 4-4-4-5-4=**25** [1] | | 88 | proving-ground | 153961 | 1 | 25.00 | 25 | read2-018 4-2-4-5-5=**25** | | 89 | screen-overlay | 152869 | 1 | 25.00 | 25 | read2-019† 4-2-5-4-5=**25** | | 90 | standing-orders | 153113 | 1 | 25.00 | 25 | deep-10† 3-3-4-5-5=**25** [4] | | 91 | submission-desk | 151397 | 1 | 25.00 | 25 | read2-022 3-5-5-4-4=**25** | | 92 | takeover | 151964 | 1 | 25.00 | 25 | deep-01 4-3-4-4-5=**25** [4] | | 93 | thoth | 150497 | 1 | 25.00 | 25 | deep-07 4-4-5-4-4=**25** [3] | | 94 | wings | 153781 | 1 | 25.00 | 25 | deep-07 4-3-4-4-5=**25** [4] | | 95 | manyhands | 150807 | 3 | 24.67 | 24–26 | deep-06† 4-4-4-4-5=**26** [4]; deep-09 4-2-4-4-5=**24** [5]; deep-16 4-3-3-4-5=**24** [4] | | 96 | ci-red-to-green | 153294 | 2 | 24.50 | 24–25 | deep-10† 4-3-5-4-4=**24** [5]; deep-16 4-4-5-4-4=**25** [2] | | 97 | lumina | 153365 | 2 | 24.00 | 24 | deep-08† 3-2-5-4-5=**24** [6]; deep-14† 3-2-5-4-5=**24** [6] | | 98 | freeze | 151245 | 1 | 24.00 | 24 | deep-16 4-4-3-5-4=**24** [3] | | 99 | genui | 152159 | 1 | 24.00 | 24 | deep-18 3-4-5-4-4=**24** [4] | | 100 | heats | 152450 | 1 | 24.00 | 24 | deep-17 4-4-4-4-4=**24** [5] | | 101 | hermes-ros | 149313 | 1 | 24.00 | 24 | deep-17 3-4-5-4-4=**24** [6] | | 102 | offgrid | 153693 | 1 | 24.00 | 24 | read2-024 3-4-3-4-5=**24** | | 103 | on-air | 153960 | 1 | 24.00 | 24 | deep-18 3-4-5-4-4=**24** [3] | | 104 | oneiros | 152876 | 1 | 24.00 | 24 | read2-018 4-4-3-3-5=**24** | | 105 | phone-autopilot | 150825 | 1 | 24.00 | 24 | read2-024 4-4-4-4-4=**24** | | 106 | precedent-log | 150678 | 1 | 24.00 | 24 | read2-020 4-3-4-3-5=**24** | | 107 | sightline | 153649 | 1 | 24.00 | 24 | read2-023† 3-4-5-4-4=**24** | | 108 | tickover | 151968 | 1 | 24.00 | 24 | read2-021† 3-4-4-5-4=**24** | | 109 | workspace | 153551 | 1 | 24.00 | 24 | read2-023† 4-3-4-3-5=**24** | | 110 | wearos | 151546 | 4 | 23.50 | 22–25 | read2-021† 3-1-4-4-5=**22**; deep-04† 3-2-4-4-5=**23** [6]; deep-10† 3-2-5-4-5=**24** [6]; deep-17 3-2-5-5-5=**25** [3] | | 111 | muster | 151448 | 2 | 23.50 | 21–26 | read2-021† 4-4-4-4-5=**26**; deep-18 4-2-4-3-4=**21** [6] | | 112 | guild | 151704 | 2 | 23.00 | 22–24 | deep-02 4-2-4-4-4=**22** [6]; deep-12† 4-1-5-4-5=**24** [6] | | 113 | agent-pool | 153225 | 1 | 23.00 | 23 | deep-09 3-2-4-4-5=**23** [6] | | 114 | annotary | 153900 | 1 | 23.00 | 23 | deep-11 4-2-5-4-4=**23** [6] | | 115 | coffee-vram | 151098 | 1 | 23.00 | 23 | read2-020 4-3-3-3-5=**23** | | 116 | cron-budget | 153071 | 1 | 23.00 | 23 | read2-022 4-3-3-3-5=**23** | | 117 | epoch | 149757 | 1 | 23.00 | 23 | read2-021† 3-4-4-4-4=**23** | | 118 | perf-lab | 151227 | 1 | 23.00 | 23 | read2-023† 4-3-5-3-4=**23** | | 119 | plugin-safehouse | 153728 | 1 | 23.00 | 23 | read2-018 3-4-4-4-4=**23** | | 120 | skilluminati | 153804 | 1 | 23.00 | 23 | read2-018 4-3-4-4-4=**23** | | 121 | clearvoice | 149978 | 2 | 22.50 | 22–23 | deep-05 3-2-5-4-4=**22** [6]; deep-16 3-3-5-4-4=**23** [5] | | 122 | egress-guard | 150913 | 1 | 22.00 | 22 | read2-024 3-4-3-4-4=**22** | | 123 | proofly | 152619 | 1 | 22.00 | 22 | read2-024 3-4-3-4-4=**22** | | 124 | slopcannon | 150208 | 1 | 22.00 | 22 | read2-018 3-3-5-3-4=**22** | | 125 | triage | 154048 | 1 | 22.00 | 22 | deep-02 4-3-3-4-4=**22** [5] | ## Each deep seat's own order (rank 1→6, total in brackets) - deep-01: kata(27) > locus-codegraph(25) > benchforge(25) > takeover(25) > hail-hermes(25) > 3ds-platform(26) - deep-02: workroom(29) > home-3d-workshop(27) > mandate(25) > neural-mesh(25) > triage(22) > guild(22) - deep-03†: tenure(29) > seedguard(28) > workroom(28) > agent-si(27) > 3ds-platform(27) > flota(26) - deep-04†: hermes-drawbridge(28) > tironian(26) > own-compute(26) > vellum(26) > flota(25) > wearos(23) - deep-05: contract-necromancer(28) > vellum(27) > salts-secrets(26) > lethe(26) > hallway-test(25) > clearvoice(22) - deep-06†: fluent(28) > lineage(28) > habits(28) > manyhands(26) > confidential-mode(26) > seedguard(25) - deep-07: confidential-mode(27) > contract-necromancer(25) > thoth(25) > wings(25) > habits(24) > hermes-drawbridge(22) - deep-08†: say-do-gate(28) > foreman(28) > undo-send(27) > hermes-design-studio(27) > summon(26) > lumina(24) - deep-09: shadow(29) > hermes-canvas(28) > flowcanvas(27) > look(26) > manyhands(24) > agent-pool(23) - deep-10†: tironian(28) > seedguard(26) > benchforge(26) > standing-orders(25) > ci-red-to-green(24) > wearos(24) - deep-11: hermes-lens(27) > workroom(26) > permission-canvas(26) > culprit(24) > lethe(24) > annotary(23) - deep-12†: trust-ladder(28) > rich-paste(27) > neural-mesh(26) > smart-dictation(26) > shadow-army(25) > guild(24) - deep-13: argus(29) > sentinel-vision(28) > twring(28) > wake-me-when(28) > googlehome(27) > hermes-colab(26) - deep-14†: breadcrumbs(29) > hermes-ui-cards(27) > googlehome(28) > pinpoint(27) > imprint(27) > lumina(24) - deep-15†: housemate(29) > omniscience(28) > twring(28) > agent-si(27) > local-scout(26) > keryx(26) - deep-16: privacy-xray(25) > ci-red-to-green(25) > freeze(24) > manyhands(24) > clearvoice(23) > omniscience(22) - deep-17: foreman(25) > contract-necromancer(25) > wearos(25) > look(25) > heats(24) > hermes-ros(24) - deep-18: breadcrumbs(28) > hermes-canvas(26) > flota(26) > on-air(24) > genui(24) > muster(21) ## Findings by plugin Plugins are in the order of the table above. For each one: **+** is what holds up, **−** is the weaknesses, and **≠** is where the seats disagree. A test count is the number the seat itself observed, not the entrant's claim. "Validate ok" means the seat's run of the catalog check passed. ### Top of this merge (mean ≥ 27.5) - **argus** (152481). Validate ok, with a warning about private imports. - Run: 5 hooks, an auxiliary task and slash/CLI commands, no tools. Status, backfill and a refused apply of an unknown proposal all behaved. 115 tests passed. [deep-13] - **+** Intent-based review and approval-gated skill changes improve the user's own agent across sessions. - **−** It may make many model calls, and private core imports in the apply path may break when Hermes upgrades. - Only one seat read it. - **housemate** (152458)†. Validate ok on all 16 checks; doctor showed 7 tools and 1 hook. [deep-15] - Run: 5 calls reached the real ControlServer and SQLite timers. A timer was created, listed and cancelled, and −1 s got a clear range error. - Tests: 569 passed, 1 skipped. 23 import-shadowing failures appeared only when run from the plugin root. - **+** Turns spare browser devices into a room-aware Hermes platform. - **−** Hands-free use needs paired browsers, STT/TTS and wake/VAD models, and no audio was verified. - One seat, and that seat is †. - **breadcrumbs** (153020). Ranked 1st in both of its deep seats. - Validate ok, with rule-9 warnings for private imports in 4 files. [deep-14]† - deep-14 configured it as memory provider and context engine: crumb search, verbatim level-1 recovery and the error paths all worked. Tests: 668 passed, 1 timing failure in the hive tests, which passed on an isolated rerun. - deep-18 could not drive its tools ("Tool does not exist", because they come from the memory provider) and judged by reading. Tests: 669 passed. - **+** Content-addressed memory that keeps exact sources, with no model calls. - **−** Private imports. It is also an exclusive replacement for the memory/context engine, a bigger commitment than adding a tool. [deep-18] - **shadow** (153955). - read2-019† ★: blinded, order-swapped A/B replay of saved turns, with priced usage and rollback. Replays are imperfect proxies, and private core dependencies limit assurance. - deep-09 (rank 1): validate ok; 6 tools, 12 hooks, 3 middleware. Only guarded refusals were exercised; no live provider comparison or model switch. - deep-09 tests: 1458 passed, 436 skipped, plus **7 watcher-thread timeout warnings**, so the seat would not call it a clean pass. Its acceptance script: 8 pass, 4 skipped (Mac-only), 0 fail. - **tenure** (153714). - read2-024 ★: earned per-action autonomy with 24 h undo, probation and a signed passport. The 5,200-line classifier and 8,000-line Desktop bundle are hard to audit. - deep-03† (rank 1): validate ok, with security-scan cautions and private imports. - In Hermes it blocked edits outside the project and blocked a write while `file:write` was on probation. - Its own real-Hermes integration script passed 30/30. - Tests: the 900 s run reached 2006 passed, 6 failed. A rerun reproduced 5 failures: dashboard reads create plugin-data, two settings tests got 200 instead of 501, and two UI latency budgets were exceeded. - **−** The shell classifier "is explicitly not a sandbox". [deep-03] - **fluent** (150353)†. Rank 1 in deep-06. - Run: it adopted git 2.43.0 (149 commands, 2,895 options). It blocked `git status --shrot` before execution with the `--short` correction, then allowed the fixed command. - Tests: 402 passed, 140 skipped. - **−** Silent for programs it has not adopted and for remote backends; a session's PATH can differ from the binary it learned. - **grimoire** (152691)† ★. Locked-down DuckDB, query receipts and a Desktop explorer. - **−** The verifier skips bare powers of ten and year-range integers, so "every number is checked" is too broad. [read2-021] - **kumi** (152258) ★. Project-aware control of Ableton/REAPER/Renoise. - **−** Setup downloads a runtime and installs DAW components; some changes cannot be undone. [read2-020] - **lineage** (149244)†. Rank 2 in deep-06. - Run: 10 observer hooks. It produced a `lineage.receipt.v1` receipt; the signature checked, and verify reported fresh, then stale after the CSV changed. - Tests: 19 Python + 2 Node passed. - **−** Inferred links are not proof of derivation, and terminal attribution uses a time window. - **no-cap** (149272) ★. Deterministic receipts from tool hooks, with no extra model. - **−** Unsupported claims are hidden by default, and an agent with shell access can fabricate evidence. [read2-022] - **retouch** (153090)† ★. Mask-constrained edits with pixel checks outside the mask. - **−** Hidden edit sessions keep the user's usual tools, so hostile text in a picture is resisted by instructions, not by a sandbox. [read2-021] - **say-do-gate** (149365)†. Rank 1 in deep-08. All 15 checks passed with no warnings; 3 hooks. - Run: in annotate mode the false claim "All tests pass" got a `[say-do]` footer citing the exit-1 receipt. - Tests: 393 passed. - **−** Covers a three-turn ledger and a limited claim grammar; in default observe mode it only nudges on the next turn. - **sentinel-vision** (153596). Rank 2 in deep-13. Doctor showed 9 tools. - Run: a camera watch rule was created and a malformed time filter was rejected. No frame or alert was observed. - Tests: 1,679 passed. - **sheets** (152421) ★. A Desktop spreadsheet with per-row research agents. - **−** Mutating tool calls are flagged after they run, not prevented by a read-only allowlist. [read2-022] - **wake-me-when** (151135). Rank 4 in deep-13. 7 tools; the watch start/stop and unknown-ID calls worked. Tests: 115 passed, 7 skipped. - **−** No sustained real-stream alert was verified, and it needs external media and speech tools. - **workroom** (152651). Three seats: 29 [1], 28† [3] and 26 [2]. - All three: validate ok; 31 tools; synthetic selftest 480/480 matches with 0 decoys accepted; 730 passed, 16 skipped. - deep-02: needs `openpyxl`, which no manifest declares (17 collection errors without it). - deep-03: staged a sourced PowerPoint deck as `awaiting_approval`, and nothing was published. - **≠** deep-11 docked Beyond and Creative (4/4): a "huge surface (31 tools, 14 hooks, ~70 routes) for a niche of Office-file users", whose benchmarks are Windows/Office-only. deep-02 gave both 5. - **googlehome** (153826). deep-13 27 [5]; deep-14† 28 [3]. - Both: 5 tools, 2 hooks; Cast status/discovery and the error paths were clean; 293 passed, 1 skipped; Desktop 37/37. - deep-13 notes warnings for private imports and raw Desktop-bridge calls. - deep-14: the speaker's microphone cannot supply input; voice uses the host microphone. - **hermes-ui-cards** (150556). read2-019† 28 ★; deep-14† 27 [2]. - deep-14 Python tests: 463 passed, 4 failed. Two are sandbox multiprocessing failures, and **two are real Python/JS validator disagreements** (hour 24:00 and an uppercase IPvFuture URL). Desktop jsdom tests: 197/197. - Both seats: stale or duplicate confirmations rely on a model warning, not a lock. - **≠** Creative 4 vs 3. ### Strong (mean 27.0–27.4) - **tironian** (151281). read2-019† 28 ★; deep-04† 26 [2]; deep-10† 28 [1]. - **≠ on Reliable (4 / 2 / 4).** - deep-04 reproduced `hermes tironian dataset missing-family` → **uncaught ValueError traceback** and capped Reliable at 2. - deep-10 ran other bad-input paths (`promote nonexistent-family`, trainer check), which refused cleanly. - read2-019 found that the pooled-pass branch of `gated_scorecard` lets an undersampled step kind serve, which weakens the advertised Wilson-bound guarantee. - Both deep seats: 104 tests passed. The demo uses scripted history and dry-run training only, so no real fine-tune was reproduced. - **twring** (153357). read2-018 26; deep-13 28 [3]; deep-15† 28 [3]. - 84 tests passed, including the adapter against a fake WebSocket. The CLI link/status paths and HTTPS enforcement worked. - The live phone path needs the Twring app, account and server, and was not exercised. - read2-018: single linked-owner conversation only. - **agent-si** (149242). read2-023† 27; deep-03† 27 [4]; deep-15† 27 [4]. All three seats are †. - Exclusive memory provider. It refused fabricated quotes and kept conflicting facts as disputed. 19 tests passed. - **−** Keyword-only recall; vendored top-level module names can collide with other plugins; private `config_schema` imports. - **vellum** (152533). read2-018 28 ★; deep-04† 26 [4]; deep-05 27 [2]. - Both deep seats: refused calls outside an authorised paper turn, and `draw` gave a bounded no-Desktop error after about 30 s. - Tests: deep-04 73 passed. deep-05 72 passed, **1 failed twice** (streamed-text performance over 2 s), and 7 headless-Chromium sandbox checks passed. - deep-04: no screenshot or demo in the pin. - **≠** Useful 3–4 and Polish 4–5. The Desktop rendering and voice sync were not verified by any seat. - **hermes-canvas** (150644). deep-09† 28 [2]; deep-18 26 [2]. - 306 tests passed. Hooks bound real `write_file` calls and check exit codes to boxes, and a cyclic update was refused. - deep-09: needed an explicit project root, although that argument is optional. - **≠** deep-18 gave Useful 4 and Beyond 4: it only helps multi-step coding in Desktop, and box checks run model-supplied shell commands. - **pinpoint** (149426). read2-021† 27 ★; deep-14† 27 [4]. - deep-14: the audit returned `desktop_unavailable` after a 4 s wait. - Tests: engine 170/170 against headless Chromium. 6 e2e errors came from a Windows PowerShell harness. - **−** Heuristic audits, and requests have no session correlation. [read2-021] - **trust-ladder** (149738). read2-024 26 ★; deep-12† 28 [1]. - deep-12: 2 tools, 5 hooks. A learned guard blocked `echo forbidden` before execution. 96 tests passed. - **−** Pattern guards can be bypassed by scripts or aliases and fail open on hook errors. - **−** The README install command uses an owner placeholder. - **≠** Useful 4 vs 5 and Beyond 4 vs 5. - **cmi** (151887)†. A native context engine with recoverable originals. Private imports and direct database reads. [read2-019] - **excubitor** (150261)†. A turn ledger for dedup, loop and spend alerts. Budget enforcement fails open on internal errors. [read2-021] - **flowcanvas** (153240)†. Rank 3 in deep-09. 7 tools; a five-node template ran in 98 ms. - Tests: 586 passed, 4 failed (sandbox semaphores); 269 JS tests passed. - **−** The headline media flows need external providers. - **helm** (150160)†. A cross-bot Desktop desk. Depends on the app's data-slot CSS selectors. [read2-021] - **hermes-design-studio** (154067)†. Rank 4 in deep-08. 7 tools; 5 calls ran. - Tests: 355 Python passed; JS 54/55, with one 60 ms timing assertion failing. - **−** Tests hard-code the author's path, and user GLSL is "not a GPU sandbox". - **hermes-lens** (153316). Rank 1 in deep-11. A Ray-Ban Meta glasses platform with tap-to-approve; secrets go through Hermes's store. - It needs the glasses, an Xcode-built iPhone app and a paid Gemini key, so **none of it was exercised**. 67 tests passed. - **home-3d-workshop** (153024). Rank 2 in deep-02. 14 tools on a simulator; start was blocked without an owner pass. Its API key comes from the secret store. - Tests: 958 passed, 1 failed (Hermes-version string). - **−** Bambu A1 only; start/stop were exercised only on the simulator. - **imprint** (153978)†. Rank 5 in deep-14. A cited answer via the API. - Tests: 692 passed, 5 failed; browser follow-up 56 passed, 3 failed. The website safety tests "remain unverified", although the logs show private-address blocking. - **kata** (150840). Rank 1 in deep-01. The demo mined a 3-step routine, which ran in 4 ms as 1 call. 58 passed, 3 skipped. - **−** Value shows only after about 3 conversations, and `kata_list` silently accepts unknown arguments. - **numogram** (151674)† ★. ESP32 crash-watch, build and flash. Full loops shown only on ESP32-P4 and a virtual C3. [read2-023] - **nvidia-switchyard** (152256) ★. Changes model routing. - **−** Subscription routing handles provider tokens through private runtime APIs, under a documented reviewer exception. [read2-022] - **project-command-center** (152899). Closes the issue → session → PR loop on a Desktop board. - **−** Writes through `gh` auth, and worktree actions run repository hooks. [read2-018] - **rich-paste** (152408)†. Rank 2 in deep-12. Request middleware interleaved images correctly and blocked loopback fetches. - Tests: 246 passed, 2 failed (network timeouts, unresolved); Node 170 passed. - **−** Clipboard tested on macOS only. - **undo-send** (152063)†. Rank 3 in deep-08. A held `git push --dry-run` was cancelled before it executed. 64 tests passed. - **−** Misses `git -C repo push` and `curl --json`; holds also delay unattended runs. - **wrenchboard** (153911) ★. An electronics-repair bench. - **−** A client of a hosted board engine; repairs and plans require **Pro**. [read2-018] ### Middle (mean 25.5–26.7) - **3ds-platform** (149202). read2-019† 27; deep-01 26 [6]; deep-03† 27 [5]. - Encrypted (Noise IK) 3DS gateway; no tools. Tests: deep-01 212 passed. deep-03 211 passed, **1 failed twice**: a link-pattern latency test took 7.9 s and 3.3 s against a 3 s limit. - **−** All seats: tiny audience. - **−** Listens on 0.0.0.0 (authenticated) and has an opt-in plaintext mode. [deep-01] - **contract-necromancer** (152908). read2-022 28; deep-05 28 [1]; deep-07 25 [2]; deep-17 25 [2]. - Freezes person-approved tests by hash, runs them at `pre_verify`, and writes receipts. deep-05 verified a receipt CONFIRMED; deep-07 and deep-17 saw MISMATCH after an edit. - Tests: 385 under unittest. deep-07 and deep-17 got all OK. deep-05 got 3 failures, all environmental (git-ignore metadata, one timing). - **≠** deep-07 and deep-17 gave Useful 4, Polish 4, Beyond 4: it helps only coding projects with tests, and nothing gates until `/necro freeze`. read2-022 and deep-05 gave 5/5/5. - **−** Same-user processes can evade it, and shell edits are caught only by regex. - **seedguard** (151543). read2-020 27; deep-03† 28 [2]; deep-06† 25 [6]; deep-10† 26 [2]. - Main-provider requests carried placeholders, not the seed or OTP. [deep-03] [deep-10] - 157 passed, 1 skipped. [deep-03] [deep-06] [deep-10] - **≠ Reliable 4/4/3/3.** deep-06 reproduced a **real miss**: a synthetic `DEMO_API_KEY` assignment right after a single quote in a shell snippet reached the recorded model request unchanged. - **−** Every seat notes that auxiliary-model paths and raw CLI history are outside the protection. - **−** deep-10's CLI e2e script could not start the gateway, so it was not counted. - **confidential-mode** (151301). deep-06† 26 [5]; deep-07 27 [1]. - Preflight refused on-host-only while vision was auto/off-tier; it locked once vision was routed locally; `web_search` was blocked without approval. [deep-06] - Tests: deep-06 325 passed, 2 failed (registry baseline). deep-07 2829 passed, 30 failed, 6 errors, mostly missing optional tinfoil/ehbp SDKs. - **−** About 31 private core imports and about 22k lines; in_process only; shell/code network traffic is not inspected. - **−** The attested tier needs a vendor key and was not tested. - **foreman** (153851). deep-08† 28 [2]; deep-17 25 [1]. - Blocked a skipped-test diff, and blocked a `git commit` / `--no-verify` through the terminal before it ran. - Tests: deep-08 380 passed, 3 failed (SARIF path truncation, one shell-timing test at 3.64 s vs 3 s). deep-17 383 passed. - **−** One private import (`tools.terminal_tool.get_session_cwd`). - **−** A nonexistent path was reported CLEAN. [deep-08] - **−** Git through `execute_code` slips past it. [deep-17] - **local-scout** (153872). read2-023† 27 ★; deep-15† 26 [5]. - Fake-provider messages held neither the synthetic email nor the card number. Tests: 282 passed, 1 failed (optional torch absent). - **−** One-way masking stops the model from using real addresses. - **shadow-army** (153524). read2-020 28 ★; deep-12† 25 [5]. - **≠ Reliable 4 vs 2.** deep-12's nested real-Hermes host suite got 28/30. It failed the nonlocal-terminal refusal and terminal-scope checks, because the plugin copies Hermes context-variable names; the outer suite got 154/157. - **−** Fingerprinting is not atomic, and git subprocesses inherit the full environment. [read2-020] - **habits** (153606). deep-06† 28 [3]; deep-07 24 [5]. - Ran reflexes with no model turn (disk-space in 0.3 s) and refused bad graphs. - Tests: deep-06 focused suite 1,949 passed; full run 3,084 passed, 2 failed before its timeout. deep-07 4929 passed, **9 failed**: 8 guard tests drifted against Hermes HEAD's private dangerous-command detector, and 1 secret-source test. - **−** Shipped or model-written reflexes can act (tidy-downloads moves files) without an explicit yes. [deep-07] - **−** The ledger can retain secrets. [deep-06] - **agentline** (152976) ★. A real phone number. Every call, text and number purchase goes through approval, and 911/premium numbers are always blocked. Paid service, US/CA only. [read2-024] - **clotho** (152582)†. Rehearse-and-apply. Apply uses non-interactive dispatch that auto-approves dangerous-command prompts. [read2-023] - **hermes-colab** (150110). Rank 6 in deep-13. 61 tests passed. The GPU path needs the user's own Google OAuth client. - **hermes-maps** (152871). Routes are straight lines, and lookups go through a throttled public service. [read2-022] - **hermes-teleoperator** (149685). Its middleware bypasses other plugins' `pre_tool_call` checks and agent guardrails, and relies on private approval APIs. [read2-022] - **joust** (151376). Races coding subagents in worktrees; shell command text is not confined to the worktree. [read2-020] - **keryx** (153853)†. Rank 6 in deep-15. Blocked 112 and malformed numbers; the simulated call completed. - Tests: 75 passed, 3 failed (assume PipeWire). Linux/Bluetooth plus a paid voice provider. - **learnloop** (153821). Tutoring state is enforced in code; quality depends on the model. [read2-022] - **own-compute** (154010)†. Rank 3 in deep-04. 15 tools; only setup and error paths were run. - Tests: 3172 passed, **11 failed**. Nine stop/cleanup tests conflict with the `not_fetched` safeguard, and the seat capped Reliable at 3. - **permission-canvas** (151341). Rank 3 in deep-11. A computer_use provider with element-level permissions. No computer_use tool was available here, so **the core path was not run**. 314 tests passed. - **plur1bus** (151604). Semantic memory provider; heavy native dependencies; Windows ARM64 needs a separate installer. [read2-020] - **roomsmith** (153724)†. The product-fetch SSRF check resolves the host, then urllib resolves it again: a **DNS-rebinding gap**. [read2-019] - **salts-secrets** (151411). Rank 3 in deep-05. Owner and public roles; it refused a password-bearing note and blocked guest terminal/memory access. - Tests: 372 passed, 5 failed; the seat saw no leak in any failure. - **smart-dictation** (151371)†. Rank 4 in deep-12. STT provider; the valid WAV returned `stt_unavailable` because no weights were cached. 88 Python + 93 Node tests passed. - **syrinx** (153938). Its outward-action matcher fails open, and held actions are approved in text, not enforced. [read2-020] - **thumbs-fix** (153796). Undo overwrites later skill edits, and message attribution can misfire. [read2-020] - **flota** (153549). deep-03† 26 [6]; deep-04† 25 [5]; deep-18 26 [3]. - **All three seats: the catalog check FAILED** (exit 1). The security scan flagged `ssh_backdoor` and `sudo_usage` at `plugin-catalog/flota.yaml:15-16`; deep-04 calls those disclosure wording, not a backdoor. - deep-03 and deep-04 also found that `ssh_guard.install` rebinds core `SSHEnvironment.__init__` (a rule-9 core override the checker missed). - Reliable is 2 everywhere. Tests: 897–899 passed. - deep-18: "would be sent back until fixed". - **benchforge** (152504). deep-01 25 [3]; deep-10† 26 [3]. - deep-01 could not finish the toolchain setup through the proxy. deep-10 did: the draft scored 20/23, the fix 22/22, the gate blocked flashing both the failing and the unapproved firmware, and 56/56 tests passed. - Niche: Arduino Uno only. - **culprit** (152914). read2-019† 27; deep-11 24 [4]. - deep-11: 29 tests passed. The timeline had not recorded a second edit yet, and the test command runs with `shell=True`. - read2-019: parallel calls can collapse into one step, and snapshots have no retention cap. - **hail-hermes** (154028). read2-023† 26; deep-01 25 [5]. - **13 private core imports** (rule 9). Tests: 371 passed, 2 failed, while the README badge claims 403 passing. - **−** LAN traffic is unencrypted. [read2-023] - **hallway-test** (152475). read2-024 26 ★; deep-05 25 [5]. - Refused file URLs and metadata IPs; the fixed demo wrote a replay. 35 tests passed. - **−** Heavy: 300–600 MB browsers and many model calls. - **look** (152918). deep-09 26 [4]; deep-17 25 [4]. - Only synthetic benchmarks and safe refusals were run (Apple Silicon Mac only). Tests: deep-09 366 passed, 15 failed; 14 of those were the sandbox refusing AF_UNIX sockets. - **−** The model can start microphone recording with an uncapped `duration_s`, and fills and navigation need no confirmation. [deep-17] - **neural-mesh** (149750). deep-02 25 [4]; deep-12† 26 [3]. - **Both seats: `doctor` FAILS.** The manifest lists the provider methods `on_pre_compress` and `on_memory_write` as hooks. - Bitemporal recall worked (current = 30 days, March = 90 days). [deep-12] 23 tests passed. - **−** Claims beyond the code: the default recall is an MD5 bag of words, not the "hybrid" the README claims; raw turn text is stored although the README says it is redacted; about 50 vendored files are unused. [deep-02] - **summon** (153861). read2-019† 25; deep-08† 26 [5]. - A real X platform adapter. 32 tests passed. The relay e2e test could not start, and no live posting was tested. - **−** The daily budget is checked per answer, not per charged post. [read2-019] ### Lower (mean ≤ 25.0) - **hermes-drawbridge** (152755). deep-04† 28 [1]; deep-07 22 [6]. **The widest split in this merge after omniscience.** - Both: the error paths worked and the Node smoke suite passed. - deep-04 credits the per-chat folder sharing as a common remote-agent need. - deep-07 says it serves only Desktop users with a remote Hermes, its README is text-only (its two screenshots are never referenced), and nothing beyond the error paths is testable. - Both note no visual demo. deep-04 notes an editor-save race the README acknowledges. - **lethe** (153302). deep-05 26 [4]; deep-11 24 [5]. - **≠ Reliable 2 vs 4.** deep-05 armed it and found that `write_file` to `memories/USER.md` **bypasses the memory veto** and survives `/lethe-end`, while status says nothing was saved. - deep-11 could not arm it (tryit cannot send slash commands). - deep-11 also notes deletion without confirmation, and the manifest prompt mentioned under "How to read this". 67 tests passed in both seats. - **omniscience** (154025). deep-15† 28 [2]; deep-16 22 [6]. **The widest split.** - deep-15: 14 observer hooks; stored events held no synthetic secrets. Tests: 793 Python + 595 frontend passed. - deep-16 found a **reviewer-list problem**: the catalog text promises "no network calls, no LLM calls, no stored credentials", yet the viewer's `insight.py` reads provider keys from `~/.hermes/auth.json` and `.env` and calls `/models` and `/chat/completions`. The viewer also has POST routes despite "no write endpoints", and the catalog yaml still has `sha TBD`. - deep-16 also saw a detached uvicorn viewer auto-start on port 8797. - deep-15 flagged only raw Desktop-bridge calls. Because deep-16's finding is about the code and deep-15 does not address it, the finding stands unrebutted. - **atomicmail** (153491)†. DKIM-checked mail gateway; sign-ins need CPU-heavy proof-of-work. [read2-019] - **check-please** (152323). Regex heuristics; stored command excerpts are not redacted; screenshots are placeholders. [read2-018] - **codecoach** (152207). Needs bubblewrap, and steps take minutes. [read2-020] - **docket** (153876)†. Trades quoting; niche; acceptance needs an owner-provided public URL. [read2-021] - **hermes-notify** (149220). Worker deployment, app sign-in and live progress are explicitly unverified. [read2-022] - **inset** (153904)†. Accuracy on real sites is unmeasured. [read2-023] - **lights-out** (153822). AWS-CLI price gate that fails closed; AWS CLI and one region only. [read2-024] - **locus-codegraph** (153687). Rank 2 in deep-01. The signature rehearsal worked: 1 of 2 call sites would break. 146 tests passed. - **−** Refuses any project under `/root` and is Python-only. - **mandate** (153050). Rank 3 in deep-02. 161 passed, 1 failed (version pin). - **−** In observe mode a job could run `hermes mandate approve --yes` itself. - **no-manual** (152309) ★. Furniture assembly; niche. [read2-024] - **phone-senses** (153363)†. Its HTTP transport signs requests but does not encrypt the responses. [read2-023] - **privacy-xray** (150696). Rank 1 in deep-16. Config audit with 0 red / 4 amber / 24 green. 79 tests passed. - **−** The `find` action sends masked snippets to the vendor. - **proving-ground** (153961). Reliable 2: it replaces callbacks in the private `_hooks` and `_middleware` registries, a **rule-9 conflict**. [read2-018] - **screen-overlay** (152869)†. Reliable 2: `daemon_env` copies the full Hermes environment, **provider keys included**, into the Qt child. [read2-019] - **standing-orders** (153113)†. Rank 4 in deep-10. - The offline replay with real CLIP weights scored 9/10, precision 1.0, recall 0.75, with 39 cache misses. That **fails its own benchmark claim** (all correct, zero misses). - Live camera is macOS-only. 170 tests passed. - **submission-desk** (151397). Narrow; its page/word checking is weaker than its format recognition suggests. [read2-022] - **takeover** (151964). Rank 4 in deep-01. - The first call attempted an **unattended root package install** (`auto_install` is on by default). - 5 private imports. Tests: 82 passed, 34 failed, 6 errors. - **thoth** (150497). Rank 3 in deep-07. 19 tools. 76 passed, 1 failed (wording). - **−** The autopilot merges after a single approval, and dashboard merges skip the approval card. - **wings** (153781). Rank 4 in deep-07. A bad revoke leaks a raw KeyError. - **−** Setup skips prompts, the agent can revoke a device without confirmation, and a password sits in a plaintext 0600 file. - **manyhands** (150807). deep-06† 26 [4]; deep-09† 24 [5]; deep-16 24 [4]. - 68 self-tests and the live integration passed. [all three] - **≠ Reliable 4/2/3.** deep-09 reproduced **fail-open**: when the audit store raises, the `pre_tool_call` wrapper returns None and the block disappears. - deep-16: an owner approval covers person + tool, not the exact arguments. - **ci-red-to-green** (153294). deep-10† 24 [5]; deep-16 25 [2]. - deep-10 demonstrated a **false proof**: with `PYTEST_ADDOPTS=-ktest_args1` a regressing fix was accepted as full-suite evidence while 2 tests actually failed. - Tests: deep-10 96 passed; deep-16 95 passed, 1 skipped. - deep-16: CI logs go verbatim into the context, and the `pre_llm_call` GitHub call has no try. - **lumina** (153365). deep-08† 24 [6]; deep-14† 24 [6]. - macOS-only: **enable failed** on Linux (PyObjC), so Reliable is capped at 2. 84 passed, 3 failed, 65 skipped. - deep-14: Guardian accepts a model-supplied `allow_original=true` without checking user approval, and the Leak Guard fails open on OCR errors. - **freeze** (151245). Rank 3 in deep-16. 62 tests passed, though the README says 31. - **−** Replayed `compute()` code runs in a child process that the README itself says is not a hard sandbox; no UI. - **genui** (152159). Rank 4 in deep-18. Four card types, Desktop only. 39 tests passed. - **heats** (152450). Rank 5 in deep-17. 211 tests passed. - **−** `heats_apply` changes the working tree with no enforced approval. - **−** The README says attempts get Hermes's coding brief, but the code gives them its own. - **hermes-ros** (149313). Rank 6 in deep-17. The interlock blocked a 5 m/s drive; 79 tests passed. - **−** `timeout_s` is uncapped. - **offgrid** (153693). SMS/USSD via Africa's Talking; sandbox-only. [read2-024] - **on-air** (153960). Rank 3 in deep-18. 24 tools. 2330 passed, 1 failed. - **oneiros** (152876). Regex-only consolidation. [read2-018] - **phone-autopilot** (150825). Needs phone-harness plus adb; Android only. [read2-024] - **precedent-log** (150678). A veto can leave a card running indefinitely. [read2-020] - **sightline** (153649)†. Solana spending grants. It **moves money**, but under human-issued grants and native approvals; it trusts the RPC and the key-holding process. [read2-023] - **tickover** (151968)†. Its own benchmarks show weak detection of subtle anomalies. [read2-021] - **workspace** (153551)†. Finalisation depends on private interfaces. [read2-023] - **wearos** (151546). read2-021† 22; deep-04† 23 [6]; deep-10† 24 [6]; deep-17 25 [3]. - **Every seat found a catalog rule-11 problem**: `tts_air.py` stores and reads an HF token in `plugin-data/hf_token`, outside Hermes's secret store. The automated check missed it. - It SIGKILLs matching processes. [read2-021] [deep-17] - deep-17: acts unasked at load (opens a listener on 127.0.0.1:42653, sets `tts.provider`, force-installs from a GitHub mirror). - 177 tests passed. Reliable 1/2/2/2. - **muster** (151448). read2-021† 26; deep-18 21 [6]. - **≠** deep-18: **enable failed**, because uv could not build the editable distribution, so its tools never loaded. - **guild** (151704). deep-02 22 [6]; deep-12† 24 [6]. - **≠ tests:** deep-02 saw 8/10 fail. deep-12 got 10/10 using the documented `cd tests` runner. - deep-12 probe: **`.env` with a synthetic `API_KEY` was pushed unchanged**, and a symlink copied an outside file. Reliable 1. - deep-02: `guild_sync` can hard-reset the library without a prompt. - **agent-pool** (153225)†. Rank 6 in deep-09. A fresh-home enable failed. Tests: 413 passed, 47 failed. Its audit falls back to an unsandboxed subprocess. - **annotary** (153900). Rank 6 in deep-11. - First use crashes with `EXDEV`: `os.replace` across filesystems. - Tokens are kept in plain 0600 files, not Hermes's store. 331 tests passed. - **coffee-vram** (151098). Admits calls on timeout; the install command has a placeholder repo. [read2-020] - **cron-budget** (153071). Some API modes and cron children are not blocked; the check and the record can race. [read2-022] - **epoch** (149757)†. Downloads an npm CLI and needs Node 22. [read2-021] - **perf-lab** (151227)†. Runs scripts with no approval gate, and its hooks fail open. [read2-023] - **plugin-safehouse** (153728). Needs macOS `sandbox-exec`. [read2-018] - **skilluminati** (153804). **Claims beyond the code:** the pin omits the unit suite it claims. [read2-018] - **clearvoice** (149978). deep-05 22 [6]; deep-16 23 [5]. - Both seats: `piper_voices.json` is absent at the pin, so 3 tests fail and non-English voice lists are empty. - deep-16: the dashboard `/speak` route trusts a caller-supplied `approved` flag. - **egress-guard** (150913). Fails open by default. [read2-024] - **proofly** (152619). Read-only check of Base transfers; narrow. [read2-024] - **slopcannon** (150208). If bypass detection raises, the plugin treats the call as attended, so **YOLO spending can pass the unattended limit**; the per-action ceiling is unlimited by default. [read2-018] - **triage** (154048). Rank 5 in deep-02. Rewrites `skills.disabled` by default. - **Claims beyond the code:** the README says skill bodies need a user tap, but the model can load and unlock any skill. ## Reviewer-list flags found in this group These match the reasons the brief says Nous's catalog reviewer sends plugins back: - **Keys read from files instead of Hermes's secret store** - wearos: `hf_token`. [all four seats] - omniscience: reads `auth.json` and `.env`. [deep-16] - annotary: tokens in 0600 files. [deep-11] - wings: password in a plaintext file. [deep-07] - screen-overlay: copies the full environment into a child process. [read2-019†] - **Moving money without approval** - slopcannon: the bypass fails open. [read2-018] - sightline and agentline: both are approval-gated, and the seats found no bypass. - **Claims beyond the code** - neural-mesh. [deep-02] - omniscience. [deep-16] - triage. [deep-02] - skilluminati. [read2-018] - hail-hermes: badge says 403 tests. [deep-01] - standing-orders: benchmark. [deep-10†] - grimoire. [read2-021†] - heats. [deep-17] - **Code nobody can read:** no seat said this outright. Size and auditability worries were raised for tenure [read2-024] and confidential-mode [deep-07]. - **Failed the catalog check:** flota. [deep-03, deep-04, deep-18] - **Failed `doctor`:** neural-mesh. [deep-02, deep-12] - **Failed `enable`:** - lumina. [deep-08, deep-14] - muster. [deep-18] - agent-pool, on a fresh home. [deep-09]