{
 "batch": "deep-17",
 "setup": "Hermes installed from source (git clone --depth 1 github.com/NousResearch/hermes-agent at fdde27e7, uv sync --locked --python 3.14 with a local uv); `hermes plugins --help` lists install, validate and doctor. Each plugin fetched at its pinned commit and run with tryit.py against Nous's scripted fake model (tests/fakes/fake_llm_provider.py): validate, enable, doctor, then planned calls. This Hermes defers plugin tools behind tool_search/tool_call, so the empty-plan 'added tools' list is empty for all six; planned calls went through tool_call and all ran. pytest, pytest-asyncio, websocket-client, pyyaml, numpy and huggingface_hub were added to the Hermes venv to run the plugins' own tests. Code was also read in full by three read-only helper reviews, checked against the runs.",
 "plugins": [
  {
   "pr": 149313,
   "name": "hermes-ros",
   "validate": "passed (hermes plugins validate --json: ok true, every check ok, no warnings)",
   "inside": "Loaded (enable and doctor OK: 9 tools, 1 pre_tool_call hook); tools are deferred behind Hermes's tool_call bridge. On the default mock backend ros_status, ros_drive(0.3 m/s, 1 s; pose moved 0.3 m), ros_get_pose all worked; ros_drive at 5 m/s for 30 s was BLOCKED by the interlock [over_speed_linear]; ros_echo on an unknown topic returned a clean error listing known topics.",
   "tests": "passed: 79 passed, 10 skipped (live rosbridge tests) with pytest from tests/ (needs pyyaml, as requirements-dev.txt says)",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 24,
   "rank": 6,
   "best": "A fail-closed pre_tool_call safety interlock (speed/duration limits, geofence, write allowlist, human-only e-stop latch) that let a safe drive through and blocked an unsafe one inside Hermes, with a no-ROS mock robot, dashboard tab and Docker demo.",
   "worst": "A niche few Hermes users have the hardware for, and model-supplied timeout_s is never capped, so in rosbridge mode a call can hang far longer than the limits suggest.",
   "blocked": null
  },
  {
   "pr": 152450,
   "name": "heats",
   "validate": "passed (ok, no warnings; at runtime this Hermes logs 'unknown manifest field(s) ignored: provides_middleware' but the plugin loads)",
   "inside": "Loaded (doctor OK: 3 tools, 3 hooks, tool_request middleware). heats_run was stopped by its own approval gate ('requires approval ... this repo is not on the trusted list') because no human was present, for both an empty task and a real one, so no race was run here (it needs real subagents and a model); heats_wait with a path-traversal run_id returned a clean 'no Heats run' error.",
   "tests": "passed: 211 passed (pytest, ~3.5 min)",
   "useful": 4,
   "reliable": 4,
   "polish": 4,
   "creative": 4,
   "beyond": 4,
   "total": 24,
   "rank": 5,
   "best": "A pre_verify referee that runs the user's real tests before the agent may finish, plus a best-of-N race of attempts in git worktrees with protected tests and a scrubbed check environment, and an honest benchmark that does not overclaim.",
   "worst": "heats_apply changes the user's working tree with no hook-enforced approval (only the tool description asks for it), the README says attempts get Hermes's coding brief when the code deliberately gives them its own, and each race costs N subagent runs.",
   "blocked": null
  },
  {
   "pr": 153851,
   "name": "foreman",
   "validate": "passed (ok; one warning: private core import tools.terminal_tool.get_session_cwd at foreman/gate.py:661, catalog rule 9)",
   "inside": "Loaded (doctor OK: 2 tools, 2 hooks; declared llm capabilities not granted non-interactively, fail closed). In a test repo where the change skipped a test, foreman_check returned BLOCKED [test-skipped] tests/test_m.py:3 with a fix; a real `git commit` through Hermes's terminal tool was blocked by its pre_tool_call gate (repo log shows no new commit); a path outside any changed repo returned CLEAN/no_tests; foreman_review returned NOT REVIEWED (no model call with auto_review=merge), as documented.",
   "tests": "passed: 383 passed, 1 skipped in tests/ (pytest); examples/cart's one failing test is the deliberately broken demo project, not the plugin",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 1,
   "best": "A deterministic, model-free gate that really blocked a commit hiding a skipped test inside Hermes, keeps merging a human-only act, and ships a Desktop pane, screenshots and a 2-minute quickstart.",
   "worst": "It relies on a private Hermes internal (catalog rule 9 warning), and by its own README a git call made through execute_code or a script slips past the gate.",
   "blocked": null
  },
  {
   "pr": 151546,
   "name": "wearos",
   "validate": "passed (ok; warnings: security scan 'caution' for sudo_usage at __init__.py:1407 and desktop/plugin.js:39-40, which only print/label a sudo suggestion; nothing runs sudo)",
   "inside": "Loaded (doctor OK: 3 tools, 1 hook, plus TTS provider; doctor printed a traceback from a network call it denies during the check, yet passed). On load it opened its claim door on 127.0.0.1:42653 unasked. watch_status returned a clear per-check report (no tailnet here, claim door answering, fix commands); watch_send with no paired watch and watch_send with no text both returned clean errors. watch_pair not run (it opens a pairing window). No watch or tailnet here, so pairing and voice were judged by reading.",
   "tests": "passed: 177 passed when run from a folder named like its mirror repo; from a folder named 'wearos' one test (test_wearos_resolver_finds_the_loaded_instance) fails on a sys.modules name clash",
   "useful": 3,
   "reliable": 2,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 25,
   "rank": 3,
   "best": "It puts Hermes on the user's own wrist: gateway-minted pairing codes a model never sees, per-agent keys over the user's own tailnet, watch notifications and cloned-voice TTS, with a Desktop pane and thorough disclosures.",
   "worst": "It acts without asking at load and on events (opens listeners, sets tts.provider in config, restarts the gateway after an unpair, force-installs code from a hard-coded GitHub mirror, SIGKILLs matching processes) and falls back to a plaintext hf_token file outside Hermes's secret store.",
   "blocked": null
  },
  {
   "pr": 152908,
   "name": "contract-necromancer",
   "validate": "passed (ok, every check ok, no warnings)",
   "inside": "Loaded (doctor OK: 2 tools, 4 hooks). necro_status and necro_check in an unfrozen project returned clear 'No frozen acceptance contracts ... type /necro freeze' messages; necro_check with a bogus argument was accepted and ignored. Its own `necro demo` (scripted, no model) ran end to end: the verify gate sent a failing fix back, passed the real fix, wrote a receipt, /necro verify CONFIRMED it and reported MISMATCH after the file changed.",
   "tests": "passed: 385 tests OK (python -m unittest discover -s tests, the CI command); pytest from the repo root errors on collection, which is not its runner",
   "useful": 4,
   "reliable": 5,
   "polish": 4,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 2,
   "best": "Freezes the project's tests out of the model's reach, makes Hermes's verify step run them and send the agent back with the failing test named, and stamps a content-hashed receipt anyone can re-check, all stdlib with a working no-key demo.",
   "worst": "Its gate only fires after write_file/patch edits or a done-claim, and shell edits to locked tests are stopped only by regex heuristics (the README admits it is not a sandbox).",
   "blocked": null
  },
  {
   "pr": 152918,
   "name": "look",
   "validate": "passed (ok; warning: private core imports hermes_yaml / hermes_cli.auth in benchmark/probe_profile.py and runtime/native_input.py, catalog rule 9)",
   "inside": "Loaded (doctor OK: 14 tools, 7 hooks, 2 middleware); on load it starts a localhost HTTP server and a speech warmup (disclosed). gaze_benchmark returned a Monte Carlo accuracy table; gaze_calibrate returned settings with chrome_connected false; gaze_read with no browser returned a clean 'No active browser task' error; gaze_wait with seconds 9 was refused by the schema (max 2). The Chrome, webcam and voice paths need an Apple Silicon Mac with Chrome and were judged by reading.",
   "tests": "partly failed: 381 tests (unittest discover, its documented runner, after installing numpy): 2 failures, 14 errors, all 14 errors from this sandbox refusing AF_UNIX sockets, 4 skipped",
   "useful": 3,
   "reliable": 3,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 25,
   "rank": 4,
   "best": "A genuinely new input channel for agents: point or look at a page element, speak, and Look resolves the target and drives Chrome through nonce-confirmed risky clicks, a closed shadow-DOM overlay and pinned model downloads.",
   "worst": "Beta only on Apple Silicon Macs; the model can start microphone recording via gaze_resolve record_voice with an uncapped duration_s, fills and navigation need no human confirmation, and two of its own tests fail.",
   "blocked": null
  }
 ]
}