{
  "batch": "deep-14",
  "setup": "Installed real NousResearch/hermes-agent source at fdde27e7dae174cb4ba842589e92508852b7dd9f with uv sync --locked --python 3.14 (Python 3.14.10); confirmed plugins install/validate/doctor. Used RUN.md tryit.py and Hermes tests/fakes/fake_llm_provider.py, fresh isolated test homes, no real accounts or model keys. Added pytest/pytest-asyncio, PyYAML, Playwright and declared document-rendering dependencies for tests. All plugin checkouts match field.json pinned SHAs; source links in sources.json. Execution findings below are local observations, not README claims.",
  "plugins": [
    {
      "validate": "Passed catalog admission (exit 0): manifests, dependencies, registration, security scan and Desktop surface all passed; rule 9 warnings identify private core imports in island.py, logview.py, seats.py and slot.py. Pin b8975645e5174edbcc00d5dc86a78e536bce938b.",
      "inside": "Loaded; enable/doctor exit 0, three hooks plus memory provider, optional ACE context engine, /island, CLI, dispatch skill and Desktop explorer. Unmodified empty plan completed; default Hermes memory does not activate the provider. For the planned run, configured memory.provider and context.engine to breadcrumbs in the runner-created test home and seeded the supplied neutral demo corpus. Real AIAgent offered mcp__crumb/mcp__crumb_find; search found tea/kettle source handles, level=1 recovered d98e0bf6 verbatim, unknown id returned an error, and nonnumeric k returned a caught conversion error. All four calls completed; no optional summarizer or IPFS peer was needed.",
      "tests": "Full suite: 668 passed, 25 skipped, 1 failed, 41 subtests passed; failure in test_hive.PassTest.test_while_others_wait_a_worker_does_not_start_a_request_that_would_come_late (busy versus aged), a timing-sensitive worker-budget assertion. Isolated rerun of the whole hive file: 27 passed, consistent with a load-sensitive timing failure.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "rank": 1,
      "best": "Exact-source recall and reversible context compression directly improve the agent itself without an embedding service or model dependency.",
      "worst": "The broad integration surface includes private Hermes imports flagged by the gate, and the hive timing test did not pass in the full suite.",
      "pr": 153020,
      "name": "breadcrumbs",
      "total": 29,
      "blocked": null
    },
    {
      "validate": "Passed: all 16 catalog checks, including capability probe, security scan, no core override and Desktop surface; no warnings. Pin 06b911e1c66908ebf340c6cb981028bc45b89c73.",
      "inside": "Loaded; enable and doctor exit 0, registering pinpoint_audit and pinpoint_pins plus a prompt section. Empty plan completed. Four scripted calls through real AIAgent: tool_describe found both tools; pins list returned a successful empty store; audit returned desktop_unavailable after its bounded 4-second acknowledgement wait with setup guidance; max_findings=0 was rejected by Hermes schema validation before invocation. No actual Desktop page was available, so visual auditing remains unverified here.",
      "tests": "Initial full pytest run lacked browser runtime libraries; after installing Playwright Chromium and its libraries, full suite had 478 passed, 2 failed, 6 errors: two conftest import collisions when suites were combined, six e2e fixture errors because their Windows Desktop harness invokes unavailable powershell. Running engine suite separately resolved both import failures: 170/170 passed against real headless Chromium. The 478 count includes those 168 earlier engine passes; counts are not additive.",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 4,
      "best": "Links measured live-page findings and element pins to the agent's fix-and-recheck loop, with unusually careful fixed/gone/not-rechecked semantics.",
      "worst": "Needs both Desktop halves and browser consent, and the real Desktop rendering path could not be exercised in this headless seat.",
      "pr": 149426,
      "name": "pinpoint",
      "total": 27,
      "blocked": null
    },
    {
      "validate": "Passed catalog admission, security scan and runtime capability checks (exit 0); enable and doctor also passed. Pin b9a23c4f7ac3ea884ebeeb0cc0184f9960911c6f, subdirectory googlehome.",
      "inside": "Loaded in real Hermes; five tools (speak, notify, status, volume, devices), two hooks, Google Home platform adapter and Desktop controls. Empty plan completed. Five planned deferred-tool calls: discovery returned ok with zero devices; status clearly reported no Cast device; empty speech returned text is required; invalid notification status and noninteger volume were refused by Hermes schema validation. Conversation completed without a crash. No physical Cast speaker, microphone or configured TTS was available; speaker playback is supported here by mock-Cast integration tests only.",
      "tests": "After adding PyYAML test dependency: 293 passed, 1 skipped in pytest, including adapter/Cast protocol, media server, discovery, cron, mute and profile-scope tests; the first run had 292 passed and one missing-PyYAML failure, resolved on rerun. Desktop loader/render/behavior harness: 37/37 checks passed.",
      "useful": 4,
      "reliable": 5,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 3,
      "best": "Turns existing Cast speakers into a profile-bound Hermes voice and task-notification platform with real protocol code, mute controls and tested delivery plumbing.",
      "worst": "The speaker microphone cannot supply input; voice requests use the host microphone, and actual hardware/TTS behavior was not verified here.",
      "pr": 153826,
      "name": "googlehome",
      "total": 28,
      "blocked": null
    },
    {
      "validate": "Passed all catalog checks and dependencies, enable and doctor exit 0; warning requires review of app-markup CSS selectors in Desktop files (the examined rules are scoped under the plugin's own .imp/.ip-ds containers). Pin a8ce37bc2f62e0c1d287a36b836a227eef243747.",
      "inside": "Loaded in real Hermes, registering imprint_write auxiliary model task and a Desktop page/API; declares no tools/hooks, so unmodified tryit empty plan correctly added no chat tools and completed. Tested five API operations through real Hermes PluginHost.asgi_request with the same scripted provider: GET /state returned starter brands; POST /sources/text ingested a neutral report; POST /ask completed a background job and returned an answer with a numbered chip pointing to the actual P1 paragraph; an empty question returned HTTP 400 Type a question first; GET /chats returned the saved cited exchange. Initial fake responses with too few followups were refused cleanly; correcting the scripted responses to match its three-item schema made the flow pass. No real model writing quality or Desktop clicks were evaluated.",
      "tests": "With declared document dependencies installed, full pytest: 692 passed, 32 skipped, 5 failed, 4857 subtests passed. Two tests assume Hermes safe web client is absent, conflicting with this real-Hermes environment; three lacked a browser. Follow-up on the two browser test files using Playwright Chromium headless shell (test-only --no-sandbox wrapper required by this sandbox): 56 passed, 3 failed. HTML sanitization/render loops passed; website private-address/redirect tests still disagreed on expected title/RuntimeError behavior. Logs show private-address blocking, so these failures do not establish an SSRF bypass; those browser guarantees remain unverified. Initial missing-dependency and browser-library failures were environment setup issues.",
      "useful": 4,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 5,
      "best": "A complete source-grounded notebook and branded publishing workspace that uses the user's own Hermes model and preserves passage-level citation evidence.",
      "worst": "Number and quote matching cannot validate a claim's meaning, and three website safety tests still failed with the available headless browser, leaving that path less certain than the verified notebook flow.",
      "pr": 153978,
      "name": "imprint",
      "total": 27,
      "blocked": null
    },
    {
      "validate": "Passed: all 16 catalog checks, security safe, declarations match registrations, Desktop SDK surface passed; no warnings. Pin c40df4d41cd5439cf4902995fd4a9e5505483f25.",
      "inside": "Loaded; enable and doctor exit 0, one hui_present tool and three hooks. Empty plan completed; four scripted calls used Hermes's deferred-tool bridge: schema discovery succeeded, choice returned numbered CLI options, confirm demo returned the action/risk/command as text without executing it, and a slider with min=10/max=0 returned a precise card.min error. Interactive Desktop clicks are covered separately by entrant tests, not by this CLI conversation.",
      "tests": "Python suite from tests/ with its supplied pytest.ini: 463 passed, 12 skipped, 4 failed. Two multiprocessing tests were blocked by the sandbox denying AF_UNIX forkserver sockets; two genuine Python/JS validation disagreements remain under Python 3.14 (24:00 schedule hour and uppercase IPvFuture image URL accepted by Python, rejected by JS). Desktop jsdom suite: 197/197 passed. Earlier root-directory invocation caused package-name import errors; supplied test root resolves them.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 3,
      "beyond": 5,
      "rank": 2,
      "best": "Makes agent questions and review work interactive across Desktop and Webapp while retaining usable text fallback and human approval boundaries.",
      "worst": "Python and Desktop validators disagree on two malformed date/URL fixtures under Python 3.14, and stale confirmations are a model warning rather than an enforced execution lock.",
      "pr": 150556,
      "name": "hermes-ui-cards",
      "total": 27,
      "blocked": null
    },
    {
      "validate": "Catalog command returned exit 0 and all 16 static/probe checks passed, but --install-deps printed an actual dependency-preparation failure: macOS PyObjC build needs /usr/bin/sw_vers, unavailable on Linux. Enable was refused (exit 1); doctor could register 10 tools/4 hooks in a probe but warned osxphotos/photoscript/PyObjC were missing. Pin aaced5beb7bdb93cf7267060bb81fb49d4762c67.",
      "inside": "Not enabled in this Linux Hermes. Both empty and four-call plans completed as conversations, but none of lumina_library_info, lumina_search, lumina_create_album or lumina_redact was available: Hermes returned unknown tool for each. This is a declared macOS-only entry, not evidence that the Mac Photos path is broken; nevertheless reliability is capped at 2 under the brief because it could not load here. Read the privacy, redaction and Leak Guard paths; no real Photos library or Apple Vision OCR was exercised.",
      "tests": "84 passed, 65 skipped, 3 failed. Failures are no-library/error-path tests needing missing osxphotos; skipped tests include the entire real Vision-based Leak Guard suite and Photos-library integration. Pure privacy classification, path/policy utilities, mocked tool registration and API tests ran; no actual OCR or redaction image verification ran.",
      "useful": 3,
      "reliable": 2,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "rank": 6,
      "best": "Combines local Photos access, verified redacted copies and outbound-tool interception to protect documents the agent could otherwise leak.",
      "worst": "Guardian accepts model-supplied allow_original=true without checking user approval, while the separate Leak Guard excludes lumina_* tools and fails open on OCR errors, weakening the advertised protection.",
      "pr": 153365,
      "name": "lumina",
      "total": 24,
      "blocked": null
    }
  ]
}