{
  "batch": "deep-32",
  "setup": "Hermes source fdde27e7dae174cb4ba842589e92508852b7dd9f cloned once; uv sync --locked --python 3.14 succeeded (Python 3.14.10). plugins --help includes install, validate and doctor. Used supplied tryit.py and Nous FakeLLMServer/AIAgent, isolated scratch Hermes homes, empty then 2–5-call plans including bad inputs. Current Hermes defers plugin tools behind tool_search/tool_call, so an empty added_by_plugin array does not mean no registration. Results below are local observations, not README claims. All plugin sources fetched at field.json SHA; no live account, model, camera or watch used. MORPH required a clean normal pinned Git install before validation, then the same Hermes AIAgent/FakeLLMServer conversation with plugins.enabled preserved; the supplied runner initial staging failure is retained in its assessment. Culprit additional positive test used a separate project cwd so snapshots excluded Hermes runtime state. Test-only dependencies were installed under scratch; standalone suites ran without Hermes source on PYTHONPATH. Public source URLs below support code/docs/provenance; run outcomes and counts are my local observations, not copied README claims. Ranking breaks equal totals by likely ongoing use across Hermes users (general voice ahead of specialist visual watching; broad undo ahead of app integration, then creative review).",
  "plugins": [
    {
      "pr": 152914,
      "name": "culprit",
      "validate": "Passed all 15 catalog checks with --install-deps; no warnings. Pinned 9dd7662ceb0d25475fac878cc1499207426c660b (repository and behavior: sources.json). Source/provenance: https://raw.githubusercontent.com/d0txyz/hermes-culprit/9dd7662ceb0d25475fac878cc1499207426c660b/README.md",
      "inside": "Loaded: enable and doctor passed; 3 tools and on_session_start/pre_tool_call/post_tool_call hooks. tryit empty chat completed; planned tool_search found all 3 tools; culprit_timeline returned baseline; show_step=999999 returned a helpful missing-step error; model bisect was safely blocked because no interactive approver was present. This was an approval refusal, not a crash; real bisect/revert exercised by its engine tests. Additional integration used a separate scratch project cwd and seeded passing calc.py: a real read_file, breaking write_file, later independent write_file and timeline recorded both changes. Owner-invoked hermes culprit found step 1 in 3 test runs, and hermes culprit revert restored calc.py while preserving later.txt; resulting files checked. The model-initiated search still correctly required approval. Code/test reference: https://raw.githubusercontent.com/d0txyz/hermes-culprit/9dd7662ceb0d25475fac878cc1499207426c660b/engine.py",
      "tests": "29 passed, 1 Windows-specific test skipped (pytest tests), including bisect, selective/conflicting revert, minimal subprocess environment, secret exclusions, restart persistence and timeouts; additional real-Hermes/owner-CLI bisect and revert passed. Test source: https://raw.githubusercontent.com/d0txyz/hermes-culprit/9dd7662ceb0d25475fac878cc1499207426c660b/tests/test_engine.py",
      "useful": 5,
      "reliable": 5,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 29,
      "rank": 1,
      "best": "Durable tool-call snapshots and real test-driven bisection make an agent's own coding mistakes diagnosable and selectively reversible.",
      "worst": "Binary search assumes a single pass-to-fail transition, and ignored dependencies or external/container changes can make recorded states unrepresentative.",
      "blocked": null
    },
    {
      "pr": 154028,
      "name": "hail-hermes",
      "validate": "Passed catalog validate --install-deps, enable and doctor. Catalog warning: 13 private Hermes core imports, including transcription, TTS config and audio helpers. Pin ade2b7b2e40c6762411e04e928e5da3bba99c185. Source/provenance: https://raw.githubusercontent.com/KeyArgo/hail-hermes/ade2b7b2e40c6762411e04e928e5da3bba99c185/README.md",
      "inside": "Loaded: 4 capability-gated device tools and hail/pocket CLI extensions, plus Desktop/dashboard source. Empty plan completed; without a hub/device the planned battery/say/haptic calls returned unavailable, without crashing. In a second real-Hermes tryit session, started the pinned real WebSocket hub and paired its test client as a simulated Wear device through the owner approval control path: search found four tools, battery returned 3900mV, notification reported delivered, double haptic succeeded, and invalid haptic enum was refused before execution. This proves the hub/control/tools path with simulated hardware, not real audio, Wear OS or Desktop rendering. Code/test reference: https://raw.githubusercontent.com/KeyArgo/hail-hermes/ade2b7b2e40c6762411e04e928e5da3bba99c185/plugin/hail_hermes/tools.py",
      "tests": "379 passed, 1 skipped in isolated test environment: plugin/tests and simulator/tests. Two earlier failures in the Hermes-installed environment concerned automatic voice backend selection; both pass with the documented standalone environment. Physical watch, Android build and speech providers not tested. Test source: https://raw.githubusercontent.com/KeyArgo/hail-hermes/ade2b7b2e40c6762411e04e928e5da3bba99c185/plugin/tests/test_tools.py",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 28,
      "rank": 2,
      "best": "A substantial voice and wearable platform gives a self-hosted Hermes session a paired physical presence, notifications and device feedback beyond anything a prompt can supply.",
      "worst": "Speech, Desktop and real-watch behavior remain unverified here, and numerous private core imports make the otherwise well-tested platform sensitive to Hermes upgrades.",
      "blocked": null
    },
    {
      "pr": 152243,
      "name": "morph",
      "validate": "Passed catalog validate --install-deps and doctor (7 tools, 3 hooks); no catalog warnings. Pin 1a3a8bce10d5d5b14bd3ae3cddc908199fe2f90c. The supplied validate-then-copy/enable flow failed editable-package staging; even a Git install after validation stayed disabled despite exit 0. A fresh pinned Git install BEFORE validation enabled successfully and was used for the successful conversation below, with fake-model config added while preserving plugins.enabled. Source/provenance: https://raw.githubusercontent.com/timioni001/morph-hermes/1a3a8bce10d5d5b14bd3ae3cddc908199fe2f90c/README.md",
      "inside": "Loaded successfully after clean normal installation: 7 tools, 3 hooks, morph slash/CLI command, system section, 3 bundled skills and dashboard backend. Ran empty then five planned calls through the same real AIAgent/Nous FakeLLMServer used by tryit, using the entry own offline CRM fixture and deterministic planner: discovery returned deals/people and role permissions; workspace planning produced revision 1 with metrics/table/chart/form; inspection read a qualified deal; flag_deal proposal remained pending with applied=false and human approval required; missing-source inspection returned a stable not_found error. No external API, write approval or browser rendering exercised. Initial unavailable-tool results belong to the failed enable path, not this successful session. Code/test reference: https://raw.githubusercontent.com/timioni001/morph-hermes/1a3a8bce10d5d5b14bd3ae3cddc908199fe2f90c/tests/conftest.py",
      "tests": "281 passed in isolated Python test environment, including live local demo-service integration, dashboard APIs and security tests; checkout folder renamed morph_repo to avoid pytest importing the plugin-root wrapper as the inner morph package. Initial missing test dependencies were installed; an earlier Hermes-installed test run had 1 environment-dependent launch-token assertion failure. Browser/Node suite not run.",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 27,
      "rank": 5,
      "best": "Schema-derived workspaces combine genuine application access with human-only approvals and read-back verification instead of letting the agent approve its own writes.",
      "worst": "The supplied validation-before-enable sequence breaks editable-package staging, and GitHub/OpenAPI usefulness still depends on connector setup and each service supporting verifiable writes.",
      "blocked": null
    },
    {
      "pr": 153647,
      "name": "riah-swipe",
      "validate": "Passed catalog validate --install-deps, enable and doctor; warning for private imports tools.interrupt.is_interrupted and tools.transcription_tools.transcribe_audio. Pin 513609d47b27a2703a2fd5bcb2fa54ce35d3d966, hermes-plugin/. Source/provenance: https://raw.githubusercontent.com/RiahStudio/riah-swipe/513609d47b27a2703a2fd5bcb2fa54ce35d3d966/hermes-plugin/README.md",
      "inside": "Loaded: 4 tools, pre_llm_call/pre_gateway_dispatch hooks and swipe command. Empty plan completed; tool_search found all four deferred tools; publishing two text options succeeded with a one-time board link, status reported 0/2 decisions, results returned both undecided cards, and an empty cards array was refused by argument validation without invoking the handler. Phone and Telegram UI not live-tested. Code/test reference: https://raw.githubusercontent.com/RiahStudio/riah-swipe/513609d47b27a2703a2fd5bcb2fa54ce35d3d966/hermes-plugin/__init__.py",
      "tests": "Documented unittest runner initially: 250 passed, 57 skipped. After adding optional Telegram dependency, full suite executed all 307: 306 passed, 1 failed (test_notes_come_back_in_swipe_order_with_how_to_read_them, reversed note order). Re-running its 44-test daily-use module passed; this is a timing/order flake, not a clean full-suite pass. Initial pytest shim collection issue avoided with documented unittest runner. Test source: https://raw.githubusercontent.com/RiahStudio/riah-swipe/513609d47b27a2703a2fd5bcb2fa54ce35d3d966/hermes-plugin/tests/test_daily_use.py",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 27,
      "rank": 6,
      "best": "A bundled phone-friendly review board and chat feedback hooks turn generated options into a durable human selection and revision loop.",
      "worst": "The full suite exposed intermittent note-order reversal, and Telegram-specific behavior plus private Hermes imports add maintenance and upgrade risk.",
      "blocked": null
    },
    {
      "pr": 152063,
      "name": "undo-send",
      "validate": "Passed catalog validate --install-deps with no warnings; enable and doctor passed. Pin 10d388c951245ceeabc1101bb8ff65e75f4f6737. Doctor reported 0 tools and 1 hook; source also registers tool_execution middleware, unsend/sendnow/undo-send commands and Desktop/dashboard surfaces. Source/provenance: https://raw.githubusercontent.com/helenn-szn/undo-send/10d388c951245ceeabc1101bb8ff65e75f4f6737/README.md",
      "inside": "Loaded middleware-only plugin: empty tryit chat completed, no added model tools expected. Three-call plan: harmless printf ran normally; HTTP POST-shaped loopback command and malformed POST-without-URL were each held for the default 20s then returned ordinary curl errors (no plugin crash). In a separate real-Hermes cancellation test with a local HTTP receiver, cancelling the pending SQLite action through the plugin Store returned cancelled_by_user and the receiver got ZERO requests; subsequent printf worked and a missing-file read reported not-found. Observed limitation: both earlier failed curl returns were nevertheless stored as sent because middleware marks any returned result successful. Desktop rendering and live gateway chat undo not tested. Code/test reference: https://raw.githubusercontent.com/helenn-szn/undo-send/10d388c951245ceeabc1101bb8ff65e75f4f6737/undo_send/hold.py",
      "tests": "64 passed (pytest from tests/), covering hold timing, cancellation, send-now, races, failure propagation, sender/chat isolation and dashboard APIs; additional real-Hermes cancellation/local receiver check passed. Test source: https://raw.githubusercontent.com/helenn-szn/undo-send/10d388c951245ceeabc1101bb8ff65e75f4f6737/tests/test_undo_send.py",
      "useful": 5,
      "reliable": 4,
      "polish": 4,
      "creative": 4,
      "beyond": 5,
      "total": 27,
      "rank": 4,
      "best": "A real tool-execution grace window can prevent irreversible agent actions, with a shared cancellation state usable from Desktop and authorized chats.",
      "worst": "Name/regex classification misses some publishing forms, and normal error returns can be misleadingly recorded as sent rather than failed.",
      "blocked": null
    },
    {
      "pr": 153596,
      "name": "sentinel-vision",
      "validate": "Passed catalog validate --install-deps with all manifest, declared capabilities, dependency, security, core-override and Desktop-surface checks; no warnings. Enable and doctor passed; isolation requires in-process because alert delivery uses live gateway post_message handles. Pin fb48bf5c835fa2e8acb74dfc05c52458b3c1194d, sentinel-vision/. Source/provenance: https://raw.githubusercontent.com/StarAtNyte/hermes-sentinel/fb48bf5c835fa2e8acb74dfc05c52458b3c1194d/sentinel-vision/README.md",
      "inside": "Loaded: 9 tools, 2 auxiliary tasks, sentinel command, unload cleanup and Desktop/dashboard code. Empty tryit plan completed. Corrected 5-call plan found tools through search; setup returned actionable model/delivery/device warnings; rules returned empty list plus persisted budget; sources without probing returned capture unavailability cleanly; invalid URL scheme returned a specific error. Additional 4-call plan successfully created a package-at-door recipe watching an empty scratch image folder, listed the saved rule with zero budget use, returned zero events, and rejected another invalid source. No actual frames/vision judgments or outbound alert delivery were exercised in Hermes; synthetic pipeline tests cover those with camera/model fakes. An initial plan mistakenly requested nonexistent sentinel_list; corrected to the registered sentinel_rules, not counted as a plugin defect. Code/test reference: https://raw.githubusercontent.com/StarAtNyte/hermes-sentinel/fb48bf5c835fa2e8acb74dfc05c52458b3c1194d/sentinel-vision/__init__.py",
      "tests": "1484 passed, 2 skipped with repository-configured importlib paths. Initial run from plugin cwd had a top-level-import assertion failure, resolved with correct cwd/import paths. The two skipped host-contract checks were separately enabled with HERMES_AGENT_DIR pointing to current Hermes: both failed because the test parser expects PluginContext in plugins.py and its vocabulary snapshot lacks on_session_open/post_curator_run; 7 accompanying loader tests passed. Device/model end-to-end tests use fakes, not a real camera or vision model. Test source: https://raw.githubusercontent.com/StarAtNyte/hermes-sentinel/fb48bf5c835fa2e8acb74dfc05c52458b3c1194d/sentinel-vision/tests/test_e2e.py",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 28,
      "rank": 3,
      "best": "A persistent, budgeted visual event loop adds cameras/screens, confirmed semantic triggers, photo event memory and alert-driven agent work that a prompt cannot keep running.",
      "worst": "Its live value depends on a capable vision model and correctly routed alerts, while Desktop/chat-photo delivery and latest cost controls lack full live verification and its host-contract test snapshot has drifted.",
      "blocked": null
    }
  ]
}