{
 "batch": "deep-40",
 "setup": "Hermes was not installed. Every shell network request from this machine was refused by the sandbox proxy (curl: (56) Proxy CONNECT aborted for gist.githubusercontent.com and github.com), so RUN.md's uv sync and tryit.py could not run. Each plugin was judged by reading its README, manifest and code at the pinned commit from field.json through a web-fetch tool. No catalog check, Hermes load, call or plugin test was run, and the scores reflect reading only.",
 "plugins": [
  {
   "pr": 153240,
   "name": "flowcanvas",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, register() adds the flow_* tools from engine tool_schemas (toolset flowcanvas), the /flow command and a bundled flowcanvas skill, plus a Desktop app (desktop/plugin.js) and a dashboard. Tool and command wrappers catch exceptions and return JSON errors.",
   "tests": "not run; the suite is large: 49 unittest files in tests/ plus JS tests in tests/js and a scripts/check.sh gate",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 5,
   "total": 27,
   "rank": 1,
   "best": "A full visual workflow canvas inside Hermes Desktop: parallel branches, per-node caching, If/Switch/For-each nodes and 12 templates. Flows run the same from the UI, /flow or cron, and write, shell and browser tools are blocked by default.",
   "worst": "Media presets depend on a third-party MCP (@nanogpt/mcp) and paid provider calls, and register() itself has no guard, so a backend import failure stops the plugin from loading.",
   "blocked": null
  },
  {
   "pr": 153938,
   "name": "syrinx",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, runtime.register adds 4 tools (syrinx_park/interrupt/status/flow), 6 hooks (pre/post_tool_call, pre/post_llm_call, on_human_input_request/resolved) and 6 commands (/flow, /catchup, /late, /night, /always, /syrinx), plus a Desktop pane.",
   "tests": "not run; tests/ has 6 pytest files (argus, backend, card, core, presence, runtime)",
   "useful": 4,
   "reliable": 3,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 27,
   "rank": 2,
   "best": "A new idea for agents: it decides when the agent's questions reach you, parks them with defaults during focus blocks, and holds outward actions (push, deploy, send) for approval on a return card.",
   "worst": "The outward-action hold fails open: if hold_directive raises, pre_tool_call logs at debug level and returns None, so the push goes through. That contradicts the claim that hooks can never break a turn.",
   "blocked": null
  },
  {
   "pr": 152159,
   "name": "genui",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, register() adds show_component, update_component and ask_user (toolset genui) and a pre_llm_call hook that sends a turn hint only on Desktop. Props are validated with jsonschema Draft 2020-12 and errors come back as {ok:false, errors}. Store file names are SHA-256-derived keys checked against ^[a-f0-9]{16}$.",
   "tests": "not run; there are no tests inside the plugin/ folder, but the repo-root tests/ (pytest, plus Playwright for the Desktop half) and an 11-task benchmark exist outside the catalog folder",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 26,
   "rank": 3,
   "best": "Live, schema-validated UI cards in Hermes Desktop (PR, task, review-list and study cards) that update in place and send button clicks back to the agent as user turns.",
   "worst": "Only renders in Desktop. show and update handlers have no try/except, and the plugin's tests and benchmark sit outside the catalog folder, with the benchmark's 100% versus 0% reported without its output.",
   "blocked": null
  },
  {
   "pr": 149611,
   "name": "hermes-security",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, plugin.register adds the security_scan_* tools (toolset security), a /security command, a hermes security-review CLI (if register_cli_command exists), recovery hooks and 7 namespaced skills, with lazy, lock-guarded service creation. A Desktop Security page ships disabled by default.",
   "tests": "not run; tests/ and evals/ folders exist and the README gives pytest -q with HERMES_AGENT_REPO set",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 3,
   "beyond": 4,
   "total": 23,
   "rank": 6,
   "best": "A careful repository security reviewer: evidence-backed findings, coverage gaps that are reported rather than hidden, secrets stored only as fingerprints, and Markdown/JSON/CSV/SARIF exports.",
   "worst": "An LLM security review is a well-known idea adapted from an upstream project. Its tests need a hermes-agent checkout on PYTHONPATH, and the README has no screenshot of the Desktop page.",
   "blocked": null
  },
  {
   "pr": 152504,
   "name": "benchforge",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, register() adds the /bench command, the bench_guide and bench_test tools, a pre_tool_call flash gate that fails closed on exceptions, and a hermes bench CLI. bench_test returns JSON errors and never raises.",
   "tests": "not run; tests/ plus 5 golden benches (10 sketches) and CI on Linux, macOS and Windows are described",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 4,
   "total": 25,
   "rank": 4,
   "best": "It tests AI-written Arduino firmware on a simulated ATmega328P with crash tests and deterministic receipts. A flash gate blocks uploads of any .hex without a passing receipt, putting Hermes on the user's own hardware.",
   "worst": "Niche (Uno only). The flash gate matches terminal commands by keyword, so other upload paths could bypass it, and progress updates sleep up to 1.4 s in-line.",
   "blocked": null
  },
  {
   "pr": 153853,
   "name": "keryx",
   "validate": "not run: Hermes could not be installed (no shell network)",
   "inside": "not run inside Hermes (no shell network); judged by reading code at the pinned commit. By reading, register() (Linux only) adds the place_call tool, pre_tool_call and agent_loop_stopped hooks, a /calls command and a hermes keryx CLI (doctor, calls, simulate). pre_tool_call runs guardrails and returns approve or block, fails closed on exceptions, and uses single-use argument-fingerprint approvals with a 15-minute TTL.",
   "tests": "not run; repo-root tests/ (outside the keryx/ catalog folder) use a fake phone, a scripted voice model and fake D-Bus/Realtime sockets",
   "useful": 3,
   "reliable": 3,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 25,
   "rank": 5,
   "best": "Hermes places real phone calls from the user's own phone over Bluetooth HFP, with per-call approval, an AI disclosure line, blocked emergency and premium numbers, quiet hours and a 5-minute cap.",
   "worst": "Heavy setup (Linux, PipeWire 1.4+, WirePlumber config, OpenAI key). load_settings reads openai_api_key from plugin config before Hermes's secret store, and the approval fingerprint is recorded before the user confirms.",
   "blocked": null
  }
 ]
}