{
 "batch": "deep-16",
 "setup": "Hermes cloned from github.com/NousResearch/hermes-agent (HEAD ff21b120) and built with uv sync --locked --python 3.14 (uv installed into test/scratch per RUN.md); tryit.py from the gist ran each plugin at its pinned commit against Hermes's scripted fake model. Plugin tools are deferred behind Hermes's tool_search/tool_call bridge, so they do not show in added_by_plugin; planned calls went through tool_call. Hook/slash-only plugins (freeze, omniscience, ci-red-to-green) were exercised with built-in terminal/read_file calls and their recorded side effects inspected; slash commands could not be driven by tryit. Plugin tests ran with Hermes's venv (unittest) or a separate venv with pytest, Hermes off PYTHONPATH.",
 "plugins": [
  {
   "pr": 151245,
   "name": "freeze",
   "validate": "passed (all checks ok, exit 0)",
   "inside": "Loaded; enable and doctor OK (0 tools, 4 hooks: pre_llm_call, post_tool_call, post_api_request, post_llm_call; /freeze slash and `hermes freeze` CLI). A chat with terminal + read_file (bad path) calls completed; the hooks recorded 1 turn and 2 calls in plugin-data/freeze/data.db. `hermes freeze list` answered 'No frozen workflows yet'; `hermes freeze run nosuch --dry-run` gave a clean 'no frozen workflow named' error. /freeze compile needs a real model (ctx.llm), not run.",
   "tests": "passed: 62 of 62 (unittest; README says 31)",
   "useful": 4,
   "reliable": 4,
   "polish": 3,
   "creative": 5,
   "beyond": 4,
   "total": 24,
   "rank": 3,
   "best": "Turns a turn the agent already did into a checked, replayable workflow that runs later with zero model calls and stops on drift, which no prompt or skill can do.",
   "worst": "Model-written compute() code runs on every replay behind an AST check and a 10 s python -I child that its own README calls not a hard sandbox; no UI or screenshots, and the README's test count is stale.",
   "blocked": null
  },
  {
   "pr": 154025,
   "name": "omniscience",
   "validate": "passed (all checks ok, exit 0); its own plugin-catalog/omniscience.yaml still has sha TBD",
   "inside": "Loaded; enable and doctor OK (0 tools, 14 hooks, /omni, dashboard tab, Desktop page). A chat with terminal + read_file calls completed; it wrote 14 events to plugin-data/omniscience/events.db and, at load, auto-started a detached uvicorn viewer on 127.0.0.1:8797. /omni not drivable via tryit.",
   "tests": "passed: 793 passed, 12 skipped (pytest, viewer paths on PYTHONPATH)",
   "useful": 4,
   "reliable": 2,
   "polish": 5,
   "creative": 3,
   "beyond": 4,
   "total": 22,
   "rank": 6,
   "best": "A thorough, heavily tested live graph and replay of every session, turn, tool call and subagent, with dashboard and Desktop pages.",
   "worst": "Its catalog text promises 'no network calls, no LLM calls, no stored credentials', yet viewer insight.py reads provider keys from ~/.hermes/auth.json and .env and calls /models and /chat/completions, and the viewer has POST routes despite 'no write endpoints': keys from files and claims beyond the code, both on the reviewer's reject list.",
   "blocked": null
  },
  {
   "pr": 149978,
   "name": "clearvoice",
   "validate": "passed (all checks ok, exit 0; --install-deps installed ptts, piper-tts, faster-whisper)",
   "inside": "Loaded; enable and doctor OK (3 tools: clearvoice_speak, clearvoice_transcribe, clearvoice_say_back; /clearvoice, CLI, dashboard API, Desktop pane). Via tool_call: speak with deliver=file returned a clear 'Writing speech to a file is off' message; transcribe of a missing file returned 'No file at ...'; speak of blank text returned 'No text to speak.' No crash; no actual audio produced in this headless run.",
   "tests": "failed: 4 failed, rest passed (about 158 passed, 1 skipped; pytest without the speech models): 3 fail because clearvoice_engine/data/piper_voices.json is gitignored (data/) and absent at the pinned commit, so non-English voice lists are empty; 1 because the local voice model was not installed",
   "useful": 3,
   "reliable": 3,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 23,
   "rank": 5,
   "best": "A careful local push-to-talk speech aid for people with speech differences, with on-device transcription, rephrasing and voices, a Desktop pane and a demo video.",
   "worst": "The bundled Piper voice index is not committed, so three of its own tests fail and voice choice for 16 non-English languages is empty, and the dashboard /speak route trusts a caller-supplied approved flag.",
   "blocked": null
  },
  {
   "pr": 150807,
   "name": "manyhands",
   "validate": "passed (all checks ok, exit 0); Hermes warns provides_middleware is an unknown manifest field",
   "inside": "Loaded; enable and doctor OK (6 tools, 3 hooks, llm_request middleware, approval transport, /many). Via tool_call: manyhands_whoami returned the owner role; manyhands_remember refused cleanly ('no person attached to this conversation'); manyhands_recall returned an empty list; recall with limit='abc' was rejected by Hermes schema validation.",
   "tests": "passed: 68 of 68 (tests/selftest.py, stdlib, stub host)",
   "useful": 4,
   "reliable": 3,
   "polish": 3,
   "creative": 4,
   "beyond": 5,
   "total": 24,
   "rank": 4,
   "best": "Makes one Hermes gateway a real multi-user assistant with per-person roles, tool allowlists, private memory, token budgets and owner-approved risky calls.",
   "worst": "Its 'fail closed' guard returns None (allow) on any exception in pre_tool_call, and an owner approval covers person plus tool rather than the exact arguments, so approving one terminal command lets the next run anything.",
   "blocked": null
  },
  {
   "pr": 150696,
   "name": "privacy-xray",
   "validate": "passed (all checks ok, exit 0); Hermes warns provides_middleware is an unknown manifest field",
   "inside": "Loaded; enable and doctor OK (1 tool privacy_xray, 2 hooks, llm_request middleware, /xray, dashboard tab). Via tool_call: check returned a 0 red / 4 amber / 24 green config audit with exact `hermes config set` fixes; map returned the leak map of the 2 requests to 127.0.0.1; find with a 2-letter query was refused as not allowed off this machine; action=delete was rejected by schema validation.",
   "tests": "passed: 79 of 79 (unittest, run from the Hermes checkout)",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 5,
   "beyond": 4,
   "total": 25,
   "rank": 1,
   "best": "Shows every user where each model request, including side tasks, actually goes and what personal data it carried, with canary tracing and a config audit that gives exact fixes it never applies itself.",
   "worst": "The find action returns masked chat snippets to the model vendor, the very leak it warns about, and there is no Desktop page or demo beyond a banner.",
   "blocked": null
  },
  {
   "pr": 153294,
   "name": "ci-red-to-green",
   "validate": "passed (all checks ok, exit 0)",
   "inside": "Loaded; enable and doctor OK (0 tools, 3 hooks, /ci, skill, dashboard routes, Desktop pane). A chat with terminal + read_file calls completed with the hooks active and no errors; /ci demo and /ci watch are slash commands tryit cannot drive, and a GitHub token was not available, so the watch-and-PR path was judged by reading.",
   "tests": "passed: 95 passed, 1 skipped (pytest; the Hermes e2e test skips without HERMES_AGENT_DIR)",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 2,
   "best": "Watches GitHub Actions and offers a fix PR only after Hermes's own test output proves the failing test failed twice before and passed three times after, with the token from the secret store and a human approval before any PR.",
   "worst": "CI log text and commit titles from the repo go verbatim into the model's context while the agent runs that repo's code, and the pre_llm_call hook makes a GitHub call outside any try.",
   "blocked": null
  }
 ]
}