{
  "batch": "deep-x-09",
  "setup": "Hermes source cloned from https://github.com/NousResearch/hermes-agent at a84025ade22d2a4085f575d4353072e5b702635d. Installed once with uv sync --locked --python 3.14; install/validate/doctor CLI commands present. Used RUN.md tryit.py with Nous tests/fakes/fake_llm_provider.py and no external model credentials. Corrected a parallel bare-home race by giving each plugin a separate report directory; disabled Hermes tool deferral in scratch configuration to expose registered tools; retained full validator JSON rather than its last 2,000 characters. Empty-plan discovery preceded planned calls. Added pytest, pytest-asyncio, PyYAML, matplotlib, Pillow and aiohttp to the scratch test interpreter as needed. All plugin checkouts remain at field.json pins. Test outcomes below are local observations, not entrant claims; no external devices/accounts were available. No judge-steering instruction was found in the reviewed entry code/docs. Close-score rank preference reflects likely retention: Augur ranks above USSD despite one fewer total point because it is usable daily by more existing Hermes users without carrier provisioning; ClearVoice ranks above Regress at the same total because its user-approval boundary held while Regress has demonstrated raw-secret capture gaps.",
  "plugins": [
    {
      "pr": 152850,
      "name": "ussd-platform",
      "validate": "Passed: hermes plugins validate --install-deps --json; all 15 admission checks passed, no warnings; enable and doctor also passed.",
      "inside": "Loaded pinned GodlyDeveloper1/hermes@f5f69e215dad54c06d2977320a080ab1c68f10ca. Adds the ussd gateway platform, not model tools (empty-plan tryit confirmed no additional tools). A separate integration used the real BasePlatformAdapter ingress and AIAgent with Nous scripted model: five HTTP calls dialled the menu (200), submitted a request (200 working screen), retrieved the actual agent answer (200), rejected a wrong webhook token (404), and rejected a malformed caller (400). One request reached the agent as Platform.USSD. No live short code, carrier or SMS account was tested. Source: https://github.com/GodlyDeveloper1/hermes/tree/f5f69e215dad54c06d2977320a080ab1c68f10ca",
      "tests": "Passed: 23 tests, including 17 engine tests and 6 live-adapter tests against installed Hermes; the initial live-module skip was resolved by installing declared aiohttp in the test interpreter. Covers pagination, delayed delivery, PIN lockout, authorization, local-only simulator and resolving a real Hermes approval entry.",
      "useful": 3,
      "reliable": 5,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "rank": 2,
      "best": "A genuine new Hermes gateway makes keypad phones usable, including paged replies and request-bound approvals that cannot approve a command the screen cannot faithfully display.",
      "worst": "Carrier provisioning is substantial adoption friction, USSD/SMS expose content to telecom intermediaries, and in-memory inboxes disappear on gateway restart (disclosed in the pinned README).",
      "total": 28,
      "blocked": null
    },
    {
      "pr": 153146,
      "name": "hermes-regress",
      "validate": "Passed: all 15 catalog admission checks, no warnings; enable and doctor passed, with four hooks and /regress registered.",
      "inside": "Loaded pinned harishkotra/hermes-regress@877c6dfdfd5e80bb1dca79a67b3081f9764545b5; adds four capture/replay hooks and /regress, no new model tools. tryit ran a valid read_file and a missing-file call; both returned normally and a real session case recorded both calls and the final answer. Additional real AIAgent replay served a recorded read and blocked an unrecorded write; the scratch marker was not created. However a recorded read limit=1 and replay limit=999 were marked exact because args_key retains path but drops limit. The public CLI replay attempt produced one error verdict at preflight with zero model requests; this machine did not provide the gateway-backed CLI runtime, so a successful end-to-end CLI replay is unverified. Source: https://github.com/harishkotra/hermes-regress/tree/877c6dfdfd5e80bb1dca79a67b3081f9764545b5",
      "tests": "Passed: all 18 entrant unit tests. Additional local probes confirmed capture, two real replay-hook calls, and no live write. A synthetic password canary passed through build_case with default redaction: the tool result was redacted, but the original prompt, assistant answer and tool arguments retained it. This is a code-confirmed privacy gap, not an external credential test.",
      "useful": 4,
      "reliable": 2,
      "polish": 3,
      "creative": 4,
      "beyond": 5,
      "rank": 6,
      "best": "Passive cases from real sessions and side-effect-vetoing replay directly address the trust lost when Hermes users change models, prompts or plugins.",
      "worst": "Despite the README claiming captured text is redacted and fixtures are argument-identical, raw prompts, answers and tool arguments retain secret-shaped text, while argument filtering can mislabel changed calls as exact.",
      "total": 23,
      "blocked": null
    },
    {
      "pr": 154055,
      "name": "augur",
      "validate": "Passed: all 16 catalog checks including Desktop SDK surface; no warnings; enable and doctor passed (4 tools, 6 hooks).",
      "inside": "Loaded pinned Bruno-coder21/augur@6e775b23e2a322c8eee32db4074a44cb85e86e89. Empty-plan discovery exposed augur_park/queue/done/report. All four planned calls ran in real Hermes: park and queue succeeded, a noninteger done ID produced a clear validation error, report generated a PNG. Additional real-agent test saved three goals via the registered /augur command, routed classification through Hermes auxiliary-task API to the scripted provider, counted two detours, and recorded a second-detour nudge in model context; this proves hook plumbing, not classifier accuracy on real models. Source: https://github.com/Bruno-coder21/augur/tree/6e775b23e2a322c8eee32db4074a44cb85e86e89",
      "tests": "100 passed, 1 failed in the full 101-test suite. The failure is the late-label latency assertion (<0.3s): 1.39s under concurrent runs and 0.324s and 0.308s on focused reruns; it is timing-sensitive and does not establish a functional failure, but the latency promise is not verified here. Friday policy/path/patch tests and privacy tests passed. No real Friday scheduler delivery or real-model classification benchmark was run.",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 1,
      "best": "Native turn hooks, token accounting, a Desktop status chip and a code-enforced research-and-draft Friday policy make weekly priorities operational rather than merely adding a reminder prompt.",
      "worst": "Every substantial interactive request adds a classifier call and the key late-label latency test failed locally, so responsiveness and real-model classification quality still need care.",
      "total": 27,
      "blocked": null
    },
    {
      "pr": 149978,
      "name": "clearvoice",
      "validate": "Passed: catalog admission including Desktop surface, no warnings; enable and doctor passed, registering 3 tools.",
      "inside": "Loaded pinned thebangarang/clearvoice-hermes-plugin@d4bc366ec72d4f2af1a1772b3a94f48bd0e0b156. Empty-plan discovery exposed speak/transcribe/say_back. Four real-Hermes calls queued a sentence with spoken=false/awaiting_user=true, rejected empty text, refused file synthesis while the user setting was off, and returned a clear missing-audio-file error. Desktop code and backend enforce user confirmation before room playback. Actual microphone recognition, voice synthesis and cloud speech were not exercised. Source: https://github.com/thebangarang/clearvoice-hermes-plugin/tree/d4bc366ec72d4f2af1a1772b3a94f48bd0e0b156",
      "tests": "158 passed, 4 failed, 1 skipped. Three failures independently demonstrate the absent clearvoice_engine/data/piper_voices.json: no non-English default voices or picker choices resolve. A fourth CLI relative-output-path test receives the missing local voice model error before path validation; the test interpreter lacks that optional speech engine, so this fourth failure is environment-dependent. The skipped test and unrun audio engines limit end-to-end confidence.",
      "useful": 3,
      "reliable": 2,
      "polish": 4,
      "creative": 4,
      "beyond": 5,
      "rank": 5,
      "best": "A privacy-conscious Desktop speech bridge gives users an editable, explicitly approved voice, with native tools that cannot force room playback.",
      "worst": "The pinned package omits the Piper voice index consumed by langs.py, breaking its advertised non-English voice selection despite the otherwise polished seventeen-language presentation.",
      "total": 23,
      "blocked": null
    },
    {
      "pr": 153960,
      "name": "on-air",
      "validate": "Passed: all 16 catalog checks including Desktop and middleware declarations, no admission warnings; enable and doctor passed. In-process discovery also logs an ignored provides_middleware manifest-field warning, but the middleware registration is real.",
      "inside": "Loaded pinned HomunculusLabs/hermes-on-air@d40b6ef97e34cd5bd89e8434cad2538a3c1080c3. Empty-plan discovery exposed all 24 owner setup/presentation tools plus the platform, hooks and runtime middleware in code. Initial four scripted calls without a host session correctly refused owner access. Repeated four calls with Hermes set_session_vars bound to a local CLI owner context: setup_status succeeded, stage_status accurately reported no configured stage, stopping an absent preview returned quiescent=true, and an invalid stage-control action was refused. No Twitch, OBS, real voice or public stream was used. Source: https://github.com/HomunculusLabs/hermes-on-air/tree/d40b6ef97e34cd5bd89e8434cad2538a3c1080c3",
      "tests": "Full suite: 2320 passed, 11 failed, 114 skipped, plus 52 passed subtests. Ten caption/adapter timing failures ALL passed on an isolated 10-test rerun (7.68s); two also passed on an earlier focused rerun. The remaining stage-lifecycle test cannot start this machine's Hermes gateway: RuntimeError gateway session bootstrap control listener unavailable. Treat that as an environment limitation, not proof of an On Air defect. Browser/device paths among skipped tests and public streaming remain unverified.",
      "useful": 3,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 3,
      "best": "A separate stage profile, real inference middleware, deterministic release guards and owner-controlled OBS/stream operations turn Hermes into a substantial guarded live co-host platform.",
      "worst": "The many moving parts require a separate configured stage and owner approvals, and a real stage startup, OBS output and voice could not be verified here.",
      "total": 26,
      "blocked": null
    },
    {
      "pr": 150453,
      "name": "workgraph",
      "validate": "Passed: catalog admission, including capability probe and all declarations. Doctor passed (1 tool, 3 hooks). Standard enable FAILED twice: uv lock could not find the staged editable workgraph-agent distribution under plugin-sources; this is an observed Hermes dependency-preparation failure, not a failed catalog gate.",
      "inside": "Pinned aliiqbal24/workgraph@37c7280a514d3fa23e7193f44b5123d1f9c90163. The required empty-plan and five-call tryit attempts were run: after enable refusal the tool was absent, two requests returned tool-not-found, and Hermes ended the conversation as an invalid tool call before the other three ran. For supplemental code assessment only, a separate scratch home manually enabled the unchanged pinned plugin and initialized a disposable Git project. Real AIAgent then completed all five calls: history, open isolated attempt, checkpoint with saved plan, invalid-action rejection, status. This bypass does not establish a working normal install. Source: https://github.com/aliiqbal24/workgraph/tree/37c7280a514d3fa23e7193f44b5123d1f9c90163",
      "tests": "Default suite: 57 passed, 12 failed (69 total). Failures share the local bubblewrap restriction: NETLINK_ROUTE socket creation is not permitted; a direct verification probe confirmed that error and fail-closed behavior. Supplemental documented WORKGRAPH_UNSAFE_LOCAL=1 on disposable fixtures: 68 passed, 1 failed; the remaining test expects the terminal guard to block, which this explicit unsafe mode disables. Thus core snapshot/merge/acceptance logic is supported, but sandbox enforcement is not verified here. Browser galleries and Chromium acceptance were not run.",
      "useful": 4,
      "reliable": 2,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 4,
      "best": "Isolated agent attempts, immutable Git checkpoints, explicit handoff state and evidence-bound candidate acceptance substantially improve Hermes coding workflows beyond what a skill can enforce.",
      "worst": "The standard Hermes enable path failed locally before the tool became available, and this sandbox cannot exercise its required bubblewrap verification boundary, so the polished workspace is not a verified normal install here.",
      "total": 25,
      "blocked": null
    }
  ]
}