{
  "batch": "deep-03",
  "setup": "Installed Hermes from source at NousResearch/hermes-agent@fdde27e7dae174cb4ba842589e92508852b7dd9f using uv sync --locked --python 3.14 (CPython 3.14.10); CLI lists install, validate and doctor. Used the supplied RUN.md and tryit.py with Nous's FakeLLMServer and AIAgent.run_conversation: empty plan, then 2–5 real tool calls per entry, including bad input. No paid model, account or physical device was used. All check results below are observations from this local run, not upstream test claims. Supplemental probes capture provider requests; agent-si is explicitly selected with memory.provider and session indexing disabled in its disposable home. Optional test dependencies and plugin dependencies were installed only in scratch. Source links in sources.json refer to the pinned code; ranks are within this six-plugin batch only. For equal totals, rank favors the plugin more Hermes users would keep installed. An initial Workroom temporary-home collision caused an enable timeout/model connection failure; separate-home rechecks passed every stage and all four calls. Supplemental direct-tool probes used the documented tools.tool_search.enabled: off setting to expose eager schemas; no plugin code was changed.",
  "plugins": [
    {
      "pr": 153714,
      "name": "tenure",
      "validate": "PASSED: catalog validate --install-deps exit 0; direct checker also passed. Security scan caution on classifier/UI strings and private-core-import warnings, not a failed gate. Enable and runtime doctor passed (7 hooks). Pin: https://github.com/otsoa-dubief/tenure/tree/f4a62855e592b9e170ab0c54d9278aabef05f370/tenure .",
      "inside": "Loaded hooks plus CLI, slash command, dashboard and Desktop UI; no new model tool. Supplied tryit plan wrote/read a disposable Python file, then conservatively blocked edits outside its current project after a read. A second five-call AIAgent probe with the fixture folder selected as cwd wrote/read valid Python, wrote/read a syntax error, then blocked the next write because file:write was on probation and no person was present. This is an expected approval denial, not a crash. The plugin's own real-Hermes integration script passed 30 checks, including actual file undo, per-type trust revocation and an intact evidence chain; its outbound/push actions were synthetic and sent nothing.",
      "tests": "Local full-suite attempt hit a 900s limit after progress showed 2006 passed, 6 failed and 1 skipped; all remaining 395 collected test nodes then passed in three tail shards (132/132/131). The six failing nodes were rerun: 5 failed, 1 passed. Reproducible issues: dashboard read creates plugin-data, two settings tests expected 501 but received 200, and two large-ledger UI latency budgets exceeded. Separately test_plugin.py passed 68/68 and the real-Hermes trusted-start integration passed 30/30 checks. Initial importlib-mode collection errors were harness configuration and resolved with normal imports.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 29,
      "rank": 1,
      "best": "Revocable autonomy, automatic probation after verified mistakes, real undo and a per-model evidence ledger improve the daily behavior of the user's own Hermes rather than adding one more task tool.",
      "worst": "Its large static shell classifier is explicitly not a sandbox, and the dashboard read-only contract has a reproducible failure despite the working core approval and undo paths.",
      "blocked": null
    },
    {
      "pr": 149202,
      "name": "3ds-platform",
      "validate": "PASSED: catalog validate --install-deps exit 0 and the same checker directly returned ok; security scan safe, no warnings. Enable and runtime doctor passed. Pin: https://github.com/rndaom/Hermes3DS/tree/3182c0b1cc97072e6aa047dd463474e3b12b84d7/plugin .",
      "inside": "Loaded a Nintendo 3DS gateway platform, device-management CLI and streaming speech provider; appropriately added no model tool. Three terminal calls inside AIAgent exercised hermes 3ds --help (pair/devices/unpair/away available), devices (clean empty-device/setup hint), and an invalid command (clear diagnostic, exit 2, no traceback). Unit tests exercised encrypted client/server pairing and messaging locally. No physical console, voice playback or Tailscale account was available, so those end-to-end paths were not verified.",
      "tests": "Local suite using normal pytest imports: 211 passed, 26 skipped, 1 failed in 38.03s. Failure: test_link_pattern_is_not_quadratic[text1], about 7.9s versus 3s. Focused repeat: 1 passed, 1 failed, same case about 3.32s. Skips include absent ffmpeg and anydoc. An earlier importlib-mode attempt caused 4 collection errors; normal mode resolved those harness errors.",
      "useful": 3,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 27,
      "rank": 5,
      "best": "A complete encrypted handheld messaging platform with native approvals, sessions, media and a real console app is an unusually creative way to extend where Hermes runs.",
      "worst": "The audience needs homebrew-capable 3DS hardware, and a hostile repeated-link input still exceeded its own latency test on both runs.",
      "blocked": null
    },
    {
      "pr": 149242,
      "name": "agent-si",
      "validate": "PASSED: catalog validate --install-deps exit 0 and direct checker passed, safe security scan; private imports from plugins.memory.config_schema are warned about. Enable and runtime doctor passed. Pin: https://github.com/joelbrilliant/hermes-agent-si/tree/adf7abc1d89b1e1ea686ac2fd29e2b95b454f4fe .",
      "inside": "Registered an exclusive memory provider, CLI and settings UI. Empty plan with the default memory provider added no tools; selecting memory.provider: agent-si exposed brain_remember, brain_search and brain_get. With an isolated brain and index_sessions false, four calls remembered an opening-date fact citing the user message, found it by keyword, retrieved its sourced value, and refused a fabricated quote. All completed without a crash. Cross-agent history indexing was reviewed in code and unit-tested with fixtures, not run against real user session files. A separate two-call boundary probe also refused value 23 December when the authentic user quote said 14 November, and brain_get confirmed no value was stored.",
      "tests": "Local pytest: 19 passed in 72.69s, including real Hermes MemoryManager lifecycle/tool routing, settings, quote gates, facts/disputes and session-schema fixtures.",
      "useful": 5,
      "reliable": 4,
      "polish": 4,
      "creative": 4,
      "beyond": 5,
      "total": 27,
      "rank": 4,
      "best": "A local, dependency-free memory provider combines source-backed facts, explicit disputes, session search and per-turn recall, and its main remember/search/get path worked inside Hermes.",
      "worst": "Recall is keyword-only, the principal attribution rule is off without setup, and vendored top-level module names plus private config-schema imports leave integration risks.",
      "blocked": null
    },
    {
      "pr": 151543,
      "name": "seedguard",
      "validate": "PASSED: catalog validate --install-deps exit 0, all 15 checks passed, safe scan, no warnings; enable and runtime doctor exit 0. Pin: https://github.com/Investorquab/Seedguard/tree/4809791d06471235fe3243fd1888c66eefb129e1 .",
      "inside": "Loaded: 5 hooks, 3 middleware registrations and /seedguard; no extra model tools. Four calls through AIAgent: read_file on a disposable public BIP39 vector returned a Seedguard placeholder; terminal with an unknown placeholder was blocked with a precise session error; missing file returned a normal not-found error; harmless terminal printf succeeded. Conversation completed. Captured main-provider requests contain the placeholder rather than the seed phrase. No live credential or funded wallet was used. Armed rehydration was not exercised in this scripted conversation; its unit tests were run.",
      "tests": "Local full suite rerun: 157 passed, 1 skipped in 10.12s. Initial concurrent run had one timing-only failure (1 MB cold scan 4.055s versus a 4s bound); the full rerun passed.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 28,
      "rank": 2,
      "best": "Actual request middleware and tool-output hooks kept a recognizable secret out of the main model context while preserving normal work and blocking unknown secret placeholders.",
      "worst": "The documented auxiliary-model and raw CLI-history gaps mean the headline protection does not cover every path by which Hermes can send a secret to a provider.",
      "blocked": null
    },
    {
      "pr": 153549,
      "name": "flota",
      "validate": "FAILED: hermes plugins validate --install-deps --json returned exit 1. Security scan: dangerous ssh_backdoor (flota.yaml:16) and sudo_usage (flota.yaml:15), from disclosure wording in plugin-catalog/flota.yaml. Direct invocation of the same catalog checker also failed. Other checks passed; private-core-import warnings. Pin: https://github.com/Bto-03/hermes-flota/tree/0b480a64f2e7229169dc3003d029f251d803a5da .",
      "inside": "Loaded and enabled; runtime doctor passed (1 tool, 2 hooks), with delegation provider, CLI, slash command, Desktop page and dashboard code. Plugin tools are deferred behind Hermes tool search, so the empty plan added no eager schemas. Four planned calls returned: status -> empty fleet; wake and off on an unconfigured machine -> clear errors; invalid action -> schema rejection without invocation. No call crashed. Physical SSH/Wake-on-LAN, remote delegation and idle sleep were not tested because no ship was supplied.",
      "tests": "Local pytest: 897 passed, 7 skipped, 1 warning in 96.89s. Tests use fake machines, not hardware.",
      "useful": 4,
      "reliable": 2,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 26,
      "rank": 6,
      "best": "Routing subagents onto sleeping personal machines, then tracking their lifecycle and showing a finished fleet UI, is a strong hardware-focused extension of Hermes itself.",
      "worst": "The catalog gate fails, and ssh_guard.install also rebinds Hermes SSHEnvironment.__init__ in violation of catalog rule 9 despite the checker missing that override.",
      "blocked": null
    },
    {
      "pr": 152651,
      "name": "workroom",
      "validate": "PASSED: catalog validate --install-deps exit 0; direct checker passed all 16 checks, safe scan and no warnings. Clean-home empty-plan recheck: validate, enable, doctor and chat all exit 0; 31 tools and 8 hook types registered. Pin: https://github.com/Pratiikpy/workroom-for-hermes/tree/c5f6bf334c1163951e203dc7c5777faadfe49795 .",
      "inside": "The supplied tryit runner passed both clean-home empty and four-call plans after the harness collision was corrected: validate/enable/doctor/chat all exit 0, with status, selftest, unknown-project error and attention completing. Registered 31 tools, 8 hook types, slash commands, dashboard and Desktop interfaces. A fresh isolated AIAgent with eager tool schemas completed status, selftest, unknown-project plan (helpful error) and attention. The synthetic selftest reported 480/480 correct matches, 0/480 accepted decoys, 300/300 split-run edits and recovery at all 15 crash points. A second five-call real-Hermes probe registered a disposable spreadsheet project, found its Q2 revenue cell, staged a new PowerPoint deck with one sourced $175 figure as awaiting_approval, rejected an unknown project and listed the pending release. The published deck did not exist afterwards. UI approval/publishing and Office rendering on real desktop applications were not exercised.",
      "tests": "All 746 collected nodes covered across a timed initial run and three tail shards: 730 passed, 16 skipped. Initial run hit 900s after 517 pass/14 skip with no failures; the remaining nodes produced 213 pass/2 skip. Separate focused Office creation/update/restore, guard, signing and recovery run: 41 passed, 1 timing-only failure (guard scan 0.133s versus 0.1s); that exact test passed separately in 0.34s. Skips include unavailable rendering/desktop components; these checks do not establish real Office-app rendering quality.",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 28,
      "rank": 3,
      "best": "Source-linked Office drafts, document-preserving edits, recovery and owner-only release approval form a substantial working product whose hooks and Desktop integration go well beyond a prompt.",
      "worst": "The broad Office feature set still depends on environment-specific rendering and owner review; the scripted draft flow was verified, while real Desktop approval and final visual quality were not.",
      "blocked": null
    }
  ]
}