{
 "batch": "deep-02",
 "setup": "Hermes cloned at fdde27e7dae1 (github.com/NousResearch/hermes-agent, depth 1) and built with uv sync --locked --python 3.14 per RUN.md; each plugin fetched at its pinned sha and run with tryit.py against Hermes's scripted fake model (empty plan, then a plan of real calls; deferred plugin tools invoked through Hermes's tool_call bridge). A local copy of tryit.py only added extra config so neural-mesh could be selected as memory.provider. pytest and the workroom deps were added to the Hermes venv to run the plugins' own tests.",
 "plugins": [
  {
   "pr": 149750,
   "name": "neural-mesh",
   "validate": "passed (exit 0); security scan 'caution' warning for destructive_root_rm patterns in a docs evidence text file, not code",
   "inside": "enabled and loaded as a memory provider once memory.provider=neural-mesh; added mesh_remember, mesh_recall, mesh_asof, mesh_stats; all 4 planned calls ran: remember stored a fact, recall returned it, mesh_asof with as_of 'not-a-date' gave a clean 'needs a parseable as_of date' error, stats reported the SQLite store. Hermes plugins doctor exit 1: ERROR unknown hook 'on_pre_compress' and 'on_memory_write' in provides_hooks (manifest lists provider methods as hooks)",
   "tests": "passed: 23 of 23 (pytest, tests/test_provider.py)",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 4,
   "best": "A real local bitemporal memory provider (as-of recall, supersession) that plugs into Hermes's memory slot with no key or network and worked on the first call.",
   "worst": "Doctor fails on hooks the manifest wrongly declares, the default recall is an MD5-hashed bag of words rather than the 'hybrid' the README claims, raw turn text is stored though the README says redacted, and ~50 vendored files (x402, bond escrow, OpenRouter) are imported but unused.",
   "blocked": null
  },
  {
   "pr": 152651,
   "name": "workroom",
   "validate": "passed (exit 0); all checks ok incl. desktop surface, 7 declared Python deps, config schema",
   "inside": "enabled; doctor OK with 31 tools and 14 hooks registered (tools served through Hermes's deferred tool_call bridge); 4 planned calls: workroom_selftest returned a synthetic battery (480/480 matches, 0 decoys accepted, 15 crash points recovered), workroom_status ready, workroom_add_project registered a folder, and workroom_explain_figure with link 'abc' was rejected cleanly by schema validation",
   "tests": "passed: 730 passed, 16 skipped (pytest), after installing openpyxl, which some test modules need but no manifest declares (17 collection errors without it)",
   "useful": 4,
   "reliable": 5,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 29,
   "rank": 1,
   "best": "Office-file provenance inside Hermes Desktop: numbers linked to workbook cells, write guards and a tripwire hook, and digest-bound human approval before publishing, all offline, with a large passing suite.",
   "worst": "Its scope is narrow to people who maintain Office decks and reports, the 31-tool surface is heavy, and accuracy figures beyond the synthetic selftest live only in bench files.",
   "blocked": null
  },
  {
   "pr": 153024,
   "name": "home-3d-workshop",
   "validate": "passed (exit 0); all checks ok",
   "inside": "enabled; doctor OK with 14 tools and 4 hooks; 5 calls via tool_call on the built-in simulator backend: home3d_status (idle simulated A1, AMS trays), home3d_find_models returned a MakerWorld card, home3d_import queued it as library file 1, home3d_slice with quality 'ultra' rejected by schema, home3d_start without a queue item/pass was BLOCKED with a clear message",
   "tests": "958 passed, 1 failed, 13 skipped (pytest); the failure (test_cli::test_plugin_keys_and_state expects 'unknown' and got 'disabled') looks like a Hermes-version difference",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 27,
   "rank": 2,
   "best": "Puts the user's own 3D printer into a group chat through Hermes, with the API key read from Hermes's secret store, a single-use owner pass and camera plate check before anything physically prints, a simulator, and a Desktop Workshop page.",
   "worst": "It serves one printer line (Bambu A1 via a Bambuddy server), the README admits start/stop were only exercised on the simulator, and one of its own tests fails on current Hermes.",
   "blocked": null
  },
  {
   "pr": 151704,
   "name": "guild",
   "validate": "passed (exit 0); all checks ok",
   "inside": "enabled; doctor OK with 4 tools and 4 hooks; without a guild, guild_status/review/sync/propose all returned clean 'not in a guild' errors; after `hermes guild init <bare repo> --solo` via the terminal tool, guild_status reported the guild, guild_sync succeeded, and both guild_propose calls (a real skill and the bad name '../Etc Passwd') were BLOCKED by its approval hook because no human was present, which is the intended gate",
   "tests": "failed: 8 of 10 failed, 2 passed (pytest); failures are 'No skill named ... in this profile' after the tests switch HERMES_HOME between members, so the suite does not hold against current Hermes",
   "useful": 4,
   "reliable": 2,
   "polish": 4,
   "creative": 4,
   "beyond": 4,
   "total": 22,
   "rank": 6,
   "best": "Shares a team's learned skills through plain git, with secret scrubbing, read-only mounted team skills, guarded edits and a human approval prompt before anything is pushed.",
   "worst": "Its own suite fails 8 of 10 here, guild_sync can hard-reset the library and drop untracked files with no prompt, and the reviewer check trusts a self-reported git user.name.",
   "blocked": null
  },
  {
   "pr": 154048,
   "name": "triage",
   "validate": "passed (exit 0); all checks ok",
   "inside": "enabled; doctor OK with 2 tools, 4 hooks plus 2 middleware; 3 calls via tool_call: skill_search on the fresh empty catalog returned no match with guidance, skill_load of an unknown skill said 'Not in the library' cleanly, and skill_search with limit 'abc' was rejected by schema; the shortlist/index-stripping middleware was read, not exercised with a populated catalog",
   "tests": "passed: 141 tests, OK (2 skipped) (unittest)",
   "useful": 4,
   "reliable": 3,
   "polish": 3,
   "creative": 4,
   "beyond": 4,
   "total": 22,
   "rank": 5,
   "best": "Replaces Hermes's full skill index in every request with a ranked shortlist and on-demand loading, which cuts prompt size for users with many skills, backed by 141 passing tests.",
   "worst": "By default it rewrites the user's Hermes skills.disabled config, and the claim that skill bodies need a user tap is contradicted by skill_load, which lets the model load and then unlock edits on any skill; there are no screenshots.",
   "blocked": null
  },
  {
   "pr": 153050,
   "name": "mandate",
   "validate": "passed (exit 0); all checks ok",
   "inside": "enabled; doctor OK with 1 tool and 2 hooks; 3 calls via tool_call: mandate_status with no job and with an unknown job both returned 'No Mandate job is bound to this run. Nothing is enforced.', and job 12345 (wrong type) was rejected by schema; pre_tool_call enforcement on a cron job was read, not exercised",
   "tests": "161 passed, 1 failed, 1 skipped (pytest); the failure is the integration test asserting the Hermes version string contains its pinned 'v0.23.0'",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 4,
   "total": 25,
   "rank": 3,
   "best": "Learns each cron job's normal tool calls, has a person approve the policy, then blocks anything outside it with a hash-checked policy and fail-closed enforcement, plus a Desktop page and a docs site.",
   "worst": "It only helps people running unattended cron jobs, and in observe mode nothing appears to stop a job running `hermes mandate approve --yes` through the terminal to approve its own policy.",
   "blocked": null
  }
 ]
}