{
 "batch": "deep-x-03",
 "setup": "Hermes installed from source as RUN.md says: git clone --depth 1 of NousResearch/hermes-agent (commit a84025a), then uv sync --locked --python 3.14 with uv's Python kept inside test/scratch; hermes plugins --help lists install, validate and doctor. Each plugin was fetched at its pinned commit from field.json and run with tryit.py against Nous's scripted test model; pytest, pytest-asyncio and duckdb were added to that environment only to run the plugins' own tests. No plugin contained text aimed at an AI judge.",
 "plugins": [
  {
   "pr": 154010,
   "name": "own-compute",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok, no warnings (manifest name cloud-compute).",
   "inside": "Loaded: enable and doctor exit 0, doctor counts 15 tools and 2 hooks; tools are reachable by name but sit behind Hermes's tool search, so the offered list did not grow. 5 of 5 planned calls ran: compute_status {} returned configured:false with next steps; compute_host types and list returned host types and an empty list; slurm_jobs {} returned an empty job list; bad input env_exec on an unknown instance returned a clean not_found error. No SSH host, Docker or AWS key here, so spawning, EC2 and the approval cards were judged by reading.",
   "tests": "failed: pytest 3177 passed, 10 failed, 30 skipped in 99 s; the 10 failures are in sandbox-stop and sudo timer tests that could not cd into temp container folders or run sudo on this machine. No CI in the repo.",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 5,
   "total": 27,
   "rank": 1,
   "best": "It moves Hermes sandboxes, Slurm jobs and agents onto machines the user already owns (SSH, Slurm, local Docker/Podman, their own EC2) with secrets read only from Hermes's secret scope, a 6,700-line readable Desktop page and a very large test suite.",
   "worst": "The spend gate for billed EC2 starts lives in a pre_tool_call approval hook rather than in the tool handlers, so with Hermes approvals switched off a model can start instances with no card (the README admits this), and the EC2-from-chat paths were not run live by the author.",
   "blocked": null
  },
  {
   "pr": 151371,
   "name": "smart-dictation",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok, no warnings; its pinned faster-whisper, av and numpy dependencies resolved and installed.",
   "inside": "Loaded: enable and doctor exit 0; it registers a transcription provider (smart_dictation) and an auxiliary correction task, 0 tools and 0 hooks, so there was no tool call to plan and the empty-plan chat completed normally. The provider and Desktop composer were judged by reading: every failure path (provider not selected, model not cached, missing or oversized audio, busy) returns a coded error and nothing downloads weights.",
   "tests": "failed: pytest 85 passed, 3 failed (the three real-PCM duration tests, which need the av package that is absent from the Hermes environment used to run them); node --test 93 tests, 87 passed, 6 skipped, 0 failed. CI in the repo runs three operating systems and rebuilds the Desktop bundle to compare.",
   "useful": 3,
   "reliable": 4,
   "polish": 3,
   "creative": 3,
   "beyond": 5,
   "total": 23,
   "rank": 5,
   "best": "A fully local speech-to-text provider for Hermes with no secrets, no network, exact-pinned dependencies, readable source for its Desktop bundle and real CI.",
   "worst": "Setup is fiddly and hard to demo: the user must set the provider by hand and already have a Whisper model cached, errors are opaque codes, and the README is one dense wall of caveats with a single screenshot.",
   "blocked": null
  },
  {
   "pr": 149640,
   "name": "hermes-ark",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok, no warnings.",
   "inside": "Loaded: enable and doctor exit 0, 1 tool, 1 hook, middleware and a CLI command. Before setup ark_checkpoint is hidden (Hermes answered that the tool does not exist) and hermes ark init without a passphrase or terminal failed cleanly, naming ARK_PASSPHRASE or --no-encryption. After hermes ark init --no-encryption: ark_checkpoint {label} returned ok with a snapshot id and restore hint; ark_checkpoint {} was refused for the missing label; the agent's terminal calls hermes ark prune and rm -rf of the repository were both BLOCKED by Ark's middleware; hermes ark list showed the snapshot (43 files, 2.8 MB) and hermes ark verify re-hashed every blob.",
   "tests": "passed: pytest 174 passed, 1 skipped in 52 s when run from a copy in an importable folder name; run in place in the hyphenated checkout folder, collection errored on the root __init__.py relative import. No CI in the repo.",
   "useful": 5,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 26,
   "rank": 2,
   "best": "Encrypted, deduplicated, verifiable snapshots of the whole Hermes home with item-level restore, taken automatically before memory and skill writes, and the agent itself is blocked from pruning, deleting or reconfiguring them.",
   "worst": "plugin.yaml declares python_dependencies: [] although the code imports the cryptography package at load, and the self-defense middleware is broad and regex-based: it fingerprints state around every non-read-only tool call and blocks any command that mentions Ark.",
   "blocked": null
  },
  {
   "pr": 150261,
   "name": "excubitor",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok, no warnings.",
   "inside": "Loaded: enable and doctor exit 0, 0 tools and 8 hooks. Planned nine identical terminal calls: calls 1 to 6 ran, call 7 was held for human approval by excubitor (no-progress loop, stagnant=6, threshold=6) and refused because no user was present, calls 8 and 9 were hard-blocked, and the log held a plain-English turn receipt (looped 8x on terminal, blocked at #8, spent $0.00 of $5 day cap). A first plan of repeated read_file calls was stopped by Hermes's own read guard before excubitor's threshold. Spend cap, inbound dedup and hang alerts need a gateway and real spend, so they were judged by reading.",
   "tests": "passed: pytest 94 passed, 2 skipped, none failed. CI in the repo runs ruff and pytest with a coverage floor.",
   "useful": 4,
   "reliable": 4,
   "polish": 3,
   "creative": 3,
   "beyond": 4,
   "total": 22,
   "rank": 6,
   "best": "One per-turn ledger behind eight hooks that really stops a no-progress tool loop inside Hermes (approval first, then a hard block) and writes a readable receipt, with a hardened webhook notifier and honest limits in the README.",
   "worst": "It fails open: any ledger error reads as zero spend so the cap silently disappears, the headline inbound dedup was not shown working in the author's own live evidence (Hermes's own idempotency rejected the duplicate), and the Desktop screenshot is a labelled mock.",
   "blocked": null
  },
  {
   "pr": 151546,
   "name": "wearos",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok; one warning, security scan caution: sudo_usage (__init__.py:1407, desktop/plugin.js:39 and :40).",
   "inside": "Loaded: enable and doctor exit 0, 3 tools and 1 hook, plus a TTS provider and CLI command. 4 of 4 planned calls ran: watch_status {} returned a full check list (not on a tailnet, tailscale binary missing, each row with a fix); watch_pair {} opened a 300-second pairing window on 127.0.0.1 and said the human types the code; watch_send with text returned a clean 'no paired watch' error; bad input watch_send {} returned 'text is required'. Enabling it also rewrote the fresh home's config.yaml to tts.provider: wearkit-gateway. No watch, tailnet or voice weights here, so pairing completion, delivery and voice were judged by reading.",
   "tests": "failed: pytest 176 passed, 1 failed in 19 s; the failure (test_wearos_resolver_finds_the_loaded_instance) asserts no module is named 'wearos', which the checkout folder name used here breaks. No CI in this mirror repo.",
   "useful": 3,
   "reliable": 3,
   "polish": 4,
   "creative": 5,
   "beyond": 5,
   "total": 25,
   "rank": 4,
   "best": "It puts Hermes on a Wear OS watch over the user's own tailnet with a carefully designed pairing (gateway-issued code a human types back, never through a model; the key released only after tailnet node binding) and adds a cloned-voice TTS provider.",
   "worst": "It does a lot at load time (sets tts.provider in the user's config, starts listener threads, runs the tailscale CLI), stores the raw 6-digit pairing code beside its hash, hands the profile's API_SERVER_KEY to paired watches over plain HTTP on the tailnet, and the watch app itself is not in the pinned repository.",
   "blocked": null
  },
  {
   "pr": 152691,
   "name": "grimoire",
   "validate": "passed: hermes plugins validate --install-deps exit 0, every check ok, no warnings.",
   "inside": "Loaded: enable and doctor exit 0, 8 tools and 6 hooks. 5 of 5 planned calls ran on a small CSV: grimoire_attach returned the dataset with typed columns; grimoire_profile returned per-column stats, roles and a correlation; grimoire_query returned correct grouped sums with a citation id; bad input DROP TABLE was refused ('Only read queries are allowed'); attaching a missing file returned 'File not found'. Its output hook also appended a one-time 'Tip: ... /grimoire learn on' line to the model's final reply.",
   "tests": "passed: python -I selftest.py printed 'selftest passed' (21 test functions, about 400 asserts, a plain script rather than pytest) once duckdb was installed beside it. CI in the repo runs it on two Python and three DuckDB versions.",
   "useful": 4,
   "reliable": 4,
   "polish": 5,
   "creative": 4,
   "beyond": 4,
   "total": 25,
   "rank": 3,
   "best": "A locked-down in-memory DuckDB (SELECT only, no external access, 30-second timeout, row cap) that lets the agent answer questions about CSV, JSON, Parquet and SQLite files and then checks the numbers in its reply against the queries that ran, finished with a Desktop page, a skill and twelve screenshots.",
   "worst": "'Every number comes from a real query' is a footer added after the fact by regex, not enforcement, and it keeps annotating unrelated numeric replies once a session has used Grimoire; duckdb is declared only in pyproject.toml, not plugin.yaml, and the token-saving benchmark cannot be reproduced without paid models.",
   "blocked": null
  }
 ]
}