{
 "batch": "deep-07",
 "setup": "Hermes cloned from github.com/NousResearch/hermes-agent (shallow, HEAD fdde27e, 2026-11-11) and built with `uv sync --locked --python 3.14` (uv installed into test/scratch; Python placed in test/scratch because the default uv Python dir was unreadable in this sandbox). tryit.py from the gist ran each plugin at its pinned commit against Hermes's scripted fake model: validate, enable, doctor, chat and planned calls. Plugin tools were not in the first tool list (Hermes offers them through tool_search, so added_by_plugin was empty for all six), but every planned call was dispatched and returned. Plugin tests ran with pytest 9.1.1 / pytest-asyncio 1.4.0 / pyyaml / jsonschema / pynacl added to the scratch venv, against this Hermes HEAD and not each plugin's own pinned Hermes.",
 "plugins": [
  {
   "pr": 151301,
   "name": "confidential-mode",
   "validate": "passed (exit 0, every check ok). Warnings: about 31 private core imports (catalog rule 9: tools.approval._lock, tools.terminal_tool._get_env_config, tools.mcp_tool._lock and others). Isolation verdict in_process, because providers.register_provider() is called directly at provider.py:1193/1274.",
   "inside": "Loaded. Enable and doctor ok: 0 tools, 11 hooks, llm/tool middleware, attested-tinfoil/attested-near provider profiles, /confidential and `hermes confidential`. Chat completed. `hermes confidential status` printed the full lock state and its limits. `check on-host-only` gave a per-route table and flagged auxiliary.vision=auto as off_tier. Bad input `on bogus` was rejected by argparse. The attested tier was not tried: it needs a Tinfoil/NEAR key.",
   "tests": "ran 5 suites (live/ excluded, it needs a NEAR key): 2829 passed, 30 failed, 6 errors, 36 skipped. Most failures are ModuleNotFoundError for the optional tinfoil/ehbp SDKs. A few seam tests fail because they pin a specific Hermes commit (84a0301) and this was HEAD.",
   "useful": 3,
   "reliable": 4,
   "polish": 5,
   "creative": 5,
   "beyond": 5,
   "total": 27,
   "rank": 1,
   "best": "It locks a whole Hermes profile to attested-enclave or on-host-only inference, through provider profiles, middleware and 11 hooks, so it changes where the model runs in a way only a plugin can, and it fails closed.",
   "worst": "It is a huge surface (about 22k lines, about 31 private core imports, in_process only) that will break on Hermes releases, and with no tools and the attested tier behind a vendor key most users cannot easily try it.",
   "blocked": null
  },
  {
   "pr": 153606,
   "name": "habits",
   "validate": "passed (exit 0, every check ok). Warning: 7 private core imports (tools.approval._presence, tools.approval_detection.detect_dangerous_command, hermes_cli.plugins.get_pre_tool_call_directive and others). Isolation: host-ready.",
   "inside": "Loaded. Doctor: 2 tools (habit_run, habit_save), 7 hooks, plus a system-prompt section and skills. habit_run {name:disk-space} ran the shipped reflex in 0.3 s with no model turn and returned df output. habit_run {} was refused with a clear issue list. habit_save with a bad name and empty fields was refused with 4 format issues. Chat completed.",
   "tests": "4929 passed, 9 failed. The failures are 8 in test_switches (the guard now flags 'chmod 600 ~/x', which looks like drift in Hermes HEAD's dangerous-command detector that habits imports privately) and 1 in test_plugin's secret-source test (the Jev decider was chosen while the test expected only llm).",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 5,
   "beyond": 4,
   "total": 24,
   "rank": 5,
   "best": "It turns the tool-call sequences Hermes repeats into reflexes that run step by step with no model turn between them, so repeat work is faster and cheaper, and you can try it in minutes with no key.",
   "worst": "Shipped or model-written reflexes can act for real (tidy-downloads moves files) without an explicit yes. Its safety guard rests on private Hermes internals, and 9 guard/secret tests already fail against current Hermes.",
   "blocked": null
  },
  {
   "pr": 153781,
   "name": "wings",
   "validate": "passed (exit 0, every check ok). Warning: 4 private core imports (hermes_cli.profiles.list_profiles, hermes_state_sessions.INTERNAL_LISTING_SOURCES). Isolation in_process, because of register_dashboard_auth_provider.",
   "inside": "Loaded from plugin/. Doctor: 2 tools (wings_status, wings_revoke_device), 0 hooks, plus a dashboard tab, a Desktop page and `hermes wings`/`/wings`. wings_status returned initialized:false cleanly. Bad input wings_revoke_device {name:nope} on an unset machine returned the raw {\"error\": \"'devices'\"} (a KeyError leaking out). The WireGuard/root setup was not run: it needs sudo and a second device.",
   "tests": "7 Python check scripts: 169 ok, 0 failed. test_root_validate.sh: 86 passed, 0 failed. These are not pytest.",
   "useful": 4,
   "reliable": 3,
   "polish": 4,
   "creative": 4,
   "beyond": 5,
   "total": 25,
   "rank": 4,
   "best": "A private WireGuard mesh with no account or coordination server joins every Hermes you own and your phone, and shows them in Desktop, the dashboard and the terminal: Hermes on your own hardware, reachable anywhere.",
   "worst": "It is root- and network-heavy (setup -y and the dashboard /start skip the prompt, the agent can revoke a device with no confirmation, a login password is kept in a plaintext 0600 file), and a bad revoke leaks a raw KeyError.",
   "blocked": null
  },
  {
   "pr": 152908,
   "name": "contract-necromancer",
   "validate": "passed (exit 0, all 15 checks ok, no warnings). Isolation: host-ready.",
   "inside": "Loaded. Doctor: 2 tools (necro_status, necro_check), 4 hooks (pre_verify, transform_llm_output, pre_tool_call, pre_llm_call), plus /necro and `hermes necro`. On the unfrozen demo copy, both tools answered cleanly ('No frozen acceptance contracts', and how a person can freeze). `hermes necro demo` played the full story: a tampered contract was blocked, the reply was stamped 'Not done.', the real fix passed with a receipt, and verify gave CONFIRMED and then MISMATCH after an edit. The tools take no arguments, so there was no bad-input call to make.",
   "tests": "unittest discover: 385 tests, OK (0 failures).",
   "useful": 4,
   "reliable": 5,
   "polish": 4,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 2,
   "best": "It hooks Hermes's verify step: the plugin itself runs the person-frozen tests and sends Hermes back until they pass, blocks edits to locked tests by hash, and writes a receipt anyone can re-check. A skill cannot enforce that.",
   "worst": "It only helps coding work in projects with tests, and nothing gates until a person runs /necro freeze from a real terminal.",
   "blocked": null
  },
  {
   "pr": 150497,
   "name": "thoth",
   "validate": "passed (exit 0, every check ok including desktop surface, no warnings). Isolation: host-ready.",
   "inside": "Loaded. Doctor: 19 tools, 3 hooks, plus a Desktop page, a dashboard API and /thoth. thoth_init (create) made a git repo and brain, and correctly refused a path inside a hidden settings folder. thoth_plan created T1, thoth_status showed the board, and thoth_note (decision) recorded N1. A bad note kind was refused by schema validation before the tool ran. thoth_assign was not run: it needs Claude Code/Codex logins.",
   "tests": "pytest: 76 passed, 1 failed, 6 skipped. The failure is test_thoth_never_makes_a_project_of_a_folder_that_holds_much_more, whose expected message 'Hermes's own' does not match the code's actual wording.",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 3,
   "best": "Hermes coordinates the real Claude Code and Codex CLIs on a shared board, each worker in its own git worktree, with cross-review, merge and plan-fingerprinted approvals, plus a polished Desktop page.",
   "worst": "A 19-tool surface, an autopilot that merges after a single plan approval, and dashboard /action merges that skip Hermes's approval card make it hard to audit, and the core path needs paid third-party CLI logins.",
   "blocked": null
  },
  {
   "pr": 152755,
   "name": "hermes-drawbridge",
   "validate": "passed (exit 0, all 16 checks ok including desktop surface, no warnings). Isolation: host-ready.",
   "inside": "Loaded. Doctor: 1 tool (drawbridge_folder), 2 hooks, plus a Desktop page and a dashboard /chat-bridge route. list_folders returned ok and []. list with nothing shared returned a clean 'offline'. request_access returned a requestId and a Desktop directive. Bad op 'delete' was refused by enum validation. Reading or writing a real share needs Hermes Desktop, so that was not run.",
   "tests": "tests/smoke.mjs under Node 24 against this Hermes checkout: 'Passed Drawbridge smoke checks against the native Hermes file API on linux' (exit 0; one assertion script, so there are no counts).",
   "useful": 3,
   "reliable": 4,
   "polish": 3,
   "creative": 4,
   "beyond": 4,
   "total": 22,
   "rank": 6,
   "best": "It lets a remote Hermes read, and propose reviewed edits to, a folder the user explicitly shares from Desktop, with exact per-op fields, approval tied to a hash of the diff shown, size limits and expiry.",
   "worst": "It only serves Desktop users with a remote Hermes, its README is text-only (its two screenshots are never referenced), and nothing beyond error paths can be tried without Desktop.",
   "blocked": null
  }
 ]
}