{
 "batch": "deep-01",
 "setup": "Hermes cloned from github.com/NousResearch/hermes-agent (depth 1) into test/scratch and built with `uv sync --locked --python 3.14` (uv and Python 3.14 installed under test/scratch); tryit.py from the gist ran each plugin against Nous's scripted fake model. Plugin tools are not offered directly but through Hermes's tool_call bridge, so planned calls went through tool_call (added_by_plugin therefore shows none for all six). Catalog check also re-run standalone (`hermes plugins validate --json`) for full output. Limits of this machine: no 3DS, Wear OS watch, Xvnc/root apt or Arduino toolchain (benchforge setup downloaded and checksum-verified arduino-cli once, then the Arduino package index fetch failed through this sandbox's proxy, and the retry download failed its checksum).",
 "plugins": [
  {
   "pr": 153687,
   "name": "locus-codegraph",
   "validate": "passed: hermes plugins validate ok, no warnings; doctor OK (5 tools, 3 hooks)",
   "inside": "loaded and enabled; registers locus_map, locus_rehearse, locus_symbol, locus_impact, locus_hotspots plus on_session_start/post_tool_call/transform_tool_result hooks. On a 2-file workspace under /root every call refused ('Refusing to index OS system directory' - /root is in its blocked roots); moved to $TMPDIR, locus_map, locus_symbol (2 call sites), locus_rehearse ('(name, *, shout=False)' -> correctly 1 of 2 call sites would break: unexpected keyword 'loud') and locus_impact (MEDIUM, 1 dependent) all worked; bad signature returned a clean 'not valid Python' error.",
   "tests": "passed: 146 unittest tests OK (its documented runner; pytest collection errors only from relative-import layout)",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 2,
   "best": "Rehearsing a Python signature change against every proven cross-file caller before editing, and flagging drift inside Hermes's own edit results, is precise and worked exactly in the run.",
   "worst": "It refuses to index any project under /root, which is where many self-hosted Hermes installs running as root keep their code, and it is limited to Python for the rehearsal features.",
   "blocked": null
  },
  {
   "pr": 150840,
   "name": "kata",
   "validate": "passed: ok, no warnings; doctor OK (1 tool, 6 hooks)",
   "inside": "loaded and enabled; registers kata_list plus post_tool_call/post_api_request/pre_tool_call/pre_llm_call/session hooks. kata_list returned an empty list with the approval note; a call with an unknown argument was accepted silently. Its scripts/demo.py ran end to end: mined a repeated 3-step routine, proposed it, approval registered kata_triage_todo, which ran 3 steps in 4 ms (1 tool call instead of 3).",
   "tests": "passed: 58 passed, 3 skipped (pytest)",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 27,
   "rank": 1,
   "best": "It watches the agent's real tool calls, mines routines repeated across conversations and, only after the user approves, compiles them into deterministic tools, which changes Hermes's own toolset in a way no prompt or skill can.",
   "worst": "Its value only shows after several sessions of repeated work (default support 3 conversations), and compiled tools appear only from the next session; kata_list itself ignores bad arguments silently.",
   "blocked": null
  },
  {
   "pr": 154028,
   "name": "hail-hermes",
   "validate": "passed with a warning: 13 private core imports (catalog rule 9, e.g. tools.tts_tool._load_tts_config, agent.auxiliary_client.call_llm); doctor OK (4 tools)",
   "inside": "loaded and enabled; registers device_notify, device_say, device_haptic, device_battery behind check_fns plus `hermes hail` CLI and a Desktop panel; with no hub or paired watch running, Hermes hid all four tools ('not available in this session'), a clean outcome. Hub and watch not run here.",
   "tests": "mostly passed: plugin 371 passed, 2 failed (test_voice_backends piper/serve defaults), 1 skipped after installing numpy; simulator 6 passed; README badge claims 403 passing",
   "useful": 3,
   "reliable": 3,
   "polish": 5,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 5,
   "best": "A voice hub that speaks answers in a Desktop panel and mirrors the same Hermes session onto a Wear OS watch paired by an approved code takes Hermes onto the user's own devices.",
   "worst": "It leans on 13 private Hermes internals the catalog warns can break on any release, two of its own tests fail, and the headline watch feature needs a Wear OS watch and sideloaded app.",
   "blocked": null
  },
  {
   "pr": 152504,
   "name": "benchforge",
   "validate": "passed: ok, no warnings; doctor OK (2 tools, 1 hook)",
   "inside": "loaded and enabled; registers bench_guide and bench_test plus a pre_tool_call flash gate. bench_guide returned the bench format; bench_test without the toolchain returned a clear 'run `hermes bench setup`' error; a non-object bench was rejected by Hermes's schema check. Setup could not complete in this sandbox (package index fetch failed), so no sketch was simulated here.",
   "tests": "passed: 56 unittest tests OK, 18 skipped (need the Arduino toolchain)",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 4,
   "total": 25,
   "rank": 3,
   "best": "It compiles AI-written Arduino firmware, runs it on a simulated ATmega328P with crash tests, and its pre_tool_call gate blocks flashing any .hex without a passing receipt for checks a person approved.",
   "worst": "It serves a niche (Arduino Uno makers), needs a ~150 MB toolchain download before anything runs, and that setup did not complete on this machine.",
   "blocked": null
  },
  {
   "pr": 149202,
   "name": "3ds-platform",
   "validate": "passed: ok, no warnings; doctor OK (platform plugin, 0 tools, 0 hooks)",
   "inside": "loaded and enabled as kind: platform (adds a '3ds' gateway platform, a piper-stream TTS provider and `hermes 3ds pair|devices|unpair|away`); it adds no agent tools, so no tool calls apply, and no 3DS was available to exercise the gateway; judged by tests and code (Noise IK auth, LAN-only pairing code).",
   "tests": "passed: 212 passed, 26 skipped (pytest in plugin/)",
   "useful": 2,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 26,
   "rank": 6,
   "best": "A real Hermes messaging platform plus a homebrew 3DS client with streaming replies, approvals, voice and camera, secured with Noise IK and LAN-only pairing, is genuinely new and well tested.",
   "worst": "Very few Hermes users own a modded Nintendo 3DS, so few would install it, and it listens on 0.0.0.0 (authenticated) with an opt-in plaintext mode.",
   "blocked": null
  },
  {
   "pr": 151964,
   "name": "takeover",
   "validate": "passed with a warning: 5 private core imports (catalog rule 9: tools.bot_desktop.*, tools.interrupt); doctor OK (1 tool, 3 hooks)",
   "inside": "loaded and enabled; registers request_takeover plus on_session_start/pre_tool_call/transform_tool_result hooks, /takeover and `hermes takeover`. The first request_takeover call tried an unattended system install (auto_install defaults to true) that failed here (installer exit 100) and returned a clear error with the apt command to give the user; a call missing 'reason' was rejected by Hermes's schema check.",
   "tests": "partly failed here: full pytest 82 passed, 34 failed, 6 errors; core+sidecar rerun with TAKEOVER_AGENT_DIR still fails mostly with screen_off / websocket 4001 from its fake Bot Desktop socket, likely environmental but not confirmed",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 4,
   "best": "Handing the agent's own browser to the person for sign-in, 2FA or payment through a live Screen view in WebUI, the dashboard or Desktop, while fencing browser tools until hand-back, is exactly the human-approval seam agents need.",
   "worst": "By default the agent's first call can start a root package install with no per-install confirmation, it imports private Bot Desktop internals, many of its tests failed here, and the README has no screenshots.",
   "blocked": null
  }
 ]
}