{
  "batch": "deep-x-08",
  "setup": "Installed Hermes from source commit a84025ade22d2a4085f575d4353072e5b702635d with uv sync --locked --python 3.14 (CPython 3.14.10); verified plugins install/validate/doctor CLI. Used the supplied tryit.py and Nous tests/fakes/fake_llm_provider.py, empty discovery plans followed by 2-4 planned calls per plugin, isolated throwaway Hermes homes, plus targeted middleware/platform integration checks. Installed pytest and optional test/plugin dependencies only under test/scratch. No real model, provider key, telco account or user secret used. Test/runtime results below are local observations; pinned source URLs identify the code reviewed, not published test reports. Tool discovery defers plugin tools: added_by_plugin was empty for all six, but heats and habits registrations were verified and planned calls routed successfully through Hermes tool_call. Scores and ranks are this seat's judgments, not official Nous results.",
  "plugins": [
    {
      "pr": 152764,
      "name": "redactgate",
      "validate": "Passed: catalog validate --install-deps --json exit 0, all 15 checks including capability declarations, security scan and no core override; enable and doctor both exit 0.",
      "inside": "Loaded: llm_request/tool_request middleware, five hooks and /redact, no new model tool. tryit: read_file returned synthetic client data to the model as PERSON/EMAIL placeholders; write_file succeeded; missing-file input returned a structured error without crashing. Extra AIAgent turn generated a write_file call using the actual EMAIL placeholder from the recorded request: the file and final reply both restored the original synthetic email. /redact verify reported four events with an intact chain, and removing an unknown term returned Not in the list. Pinned code: https://raw.githubusercontent.com/ctmakc/hermes-redactgate/d4bb87d78381d0514cd8dd74525fea98a5b46d62/engine.py",
      "tests": "Passed: 63 repository tests, including four real PluginManager/middleware integration tests; 0 failures. Extra end-to-end placeholder restoration and audit checks passed.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 28,
      "rank": 2,
      "best": "It adds practical, offline privacy at the model boundary while preserving useful local tools, with unusually clear disclosures and verified round-trip restoration.",
      "worst": "Auxiliary title/compression calls bypass request middleware, require explicit hardening or local configuration, and unusual names without cues remain undetected (documented limits: https://raw.githubusercontent.com/ctmakc/hermes-redactgate/d4bb87d78381d0514cd8dd74525fea98a5b46d62/README.md).",
      "blocked": null
    },
    {
      "pr": 152063,
      "name": "undo-send",
      "validate": "Passed: catalog validate --install-deps --json, enable and doctor all exit 0; declared capabilities, security scan, Desktop surface and no core override passed.",
      "inside": "Loaded: tool_execution middleware, pre_gateway_dispatch hook, /unsend, /sendnow and /undo-send; Desktop chip/dashboard source present, no new model tool. tryit: git status returned immediately, git push --dry-run was held for about 20 seconds then returned the expected no-push-destination error, and an invalid Git command returned its normal error. A separate real Hermes middleware-chain check with a one-second window showed a pending action, cancelled it through /unsend all and proved its downstream callback never ran; expiry executed once and a read bypassed the hold. The initial terminal plan used unsupported cwd; the supplemental checks did not depend on that argument. Pinned code: https://raw.githubusercontent.com/helenn-szn/undo-send/10d388c951245ceeabc1101bb8ff65e75f4f6737/undo_send/hold.py",
      "tests": "Passed: 64 repository tests, 0 failures, including dashboard API, classifier, holding, cancellation and chat-origin checks; supplemental cancellation/expiry/read-bypass checks passed.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 28,
      "rank": 3,
      "best": "A cancellable grace window for irreversible tool actions improves everyday agent use through middleware, Desktop controls and chat-aware cancellation.",
      "worst": "Safety coverage depends on action-name and shell heuristics, middleware deliberately fails open, and same-chat cancellation depends on Hermes gateway dispatch behavior (disclosed in https://raw.githubusercontent.com/helenn-szn/undo-send/10d388c951245ceeabc1101bb8ff65e75f4f6737/README.md).",
      "blocked": null
    },
    {
      "pr": 153146,
      "name": "hermes-regress",
      "validate": "Passed: catalog validate --install-deps --json, enable and doctor all exit 0, including security scan and no core override.",
      "inside": "Loaded: pre_tool_call replay interception, post_tool_call/post_llm_call/on_session_end capture hooks and /regress, no model tool. tryit completed a terminal printf, a file read and a missing-file error and wrote a captured case. Real registered capture-hook check with a synthetic password confirmed tool-result redaction but plaintext persistence in prompt, answer and tool arguments. /regress status worked; malformed run --limit returned a caught ValueError message. An additional armed-fixture AIAgent turn served the recorded terminal output through the real pre_tool_call hook and blocked an unrecorded write_file without creating the file. Native replay preflight could not start this environment's Hermes gateway, sent no model requests and refused replay cleanly; a successful full native replay is therefore not claimed. Pinned code: https://raw.githubusercontent.com/harishkotra/hermes-regress/877c6dfdfd5e80bb1dca79a67b3081f9764545b5/rg_capture.py",
      "tests": "Passed: all 18 repository tests, 0 failures. Supplemental real capture/privacy probe exposed unredacted synthetic credentials in three stored fields. In-process fixture replay passed: exact fixture hit, unrecorded write blocked, no file created. Native CLI replay was attempted but blocked by gateway startup failure; its preflight made no model request.",
      "useful": 4,
      "reliable": 2,
      "polish": 3,
      "creative": 4,
      "beyond": 5,
      "total": 23,
      "rank": 6,
      "best": "Passive session capture, tool-fixture interception and exportable regression reports address a valuable model-upgrade problem at Hermes runtime seams.",
      "worst": "Despite the README claiming captured text is redacted before disk, build_case redacts only tool results and persists synthetic passwords unchanged in prompts, answers and arguments (https://raw.githubusercontent.com/harishkotra/hermes-regress/877c6dfdfd5e80bb1dca79a67b3081f9764545b5/rg_capture.py).",
      "blocked": null
    },
    {
      "pr": 152850,
      "name": "ussd-platform",
      "validate": "Passed: catalog validate --install-deps --json, enable and doctor all exit 0; all 15 admission checks passed; missing real telco credentials did not prevent registration.",
      "inside": "Loaded: registered the native ussd gateway platform, no agent tool or hook. tryit agent conversation and ordinary read/error smoke calls completed. Extra test created the adapter through the real Hermes platform registry and connected its HTTP listener with temporary synthetic callback credentials: dial returned the home menu; a phone-form question traversed an actual AIAgent turn against Nous's scripted model and returned CON 42 is the answer; malformed phone returned HTTP 400, wrong secret 404, and invalid outbound chat id returned a structured SendResult error. Real operator/aggregator routing and SMS delivery were not tested. Pinned code: https://raw.githubusercontent.com/GodlyDeveloper1/hermes/f5f69e215dad54c06d2977320a080ab1c68f10ca/adapter.py",
      "tests": "Passed: 23 repository tests, 0 failures, covering engine paging, PIN/approval paths and real-Hermes adapter HTTP tests; extra registry-to-AIAgent HTTP round trip and three bad-input checks passed.",
      "useful": 3,
      "reliable": 5,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 28,
      "rank": 4,
      "best": "It brings the user's own Hermes to keypad phones through a complete platform adapter with readable approval screens, paging, reconnect behavior and a local simulator.",
      "worst": "Actual handset access needs an aggregator and paid short code, carriers can read USSD/SMS, and inbox/session state is lost on gateway restart (clearly disclosed in https://raw.githubusercontent.com/GodlyDeveloper1/hermes/f5f69e215dad54c06d2977320a080ab1c68f10ca/README.md).",
      "blocked": null
    },
    {
      "pr": 152450,
      "name": "heats",
      "validate": "Passed: catalog validate --install-deps --json, enable and doctor all exit 0; security scan, declared tools/hooks/middleware, Desktop surface and no core override passed.",
      "inside": "Loaded: heats_run/heats_wait/heats_apply (deferred discovery), three hooks, tool_request middleware, slash/CLI commands and Desktop pane source. Initial tryit: an untrusted-repo run required approval and was refused cleanly without an interactive owner; unknown run ids returned errors; n=9 was rejected by the tool schema. Extra real AIAgent/lifecycle run in a trusted synthetic Git repo with an owner-configured check: baseline failed, one real Hermes child read and changed answer.py in an isolated worktree, check then reported 1 passed, heat 1 won, original checkout stayed unchanged until the registered /heats apply command applied the saved patch. Attempts that skipped reading before writing were correctly refused by Hermes and produced no winner; the final successful run followed that prerequisite. Pinned code: https://raw.githubusercontent.com/EconLearn/hermes-heats/2bcee10e9ca6489b7fd23bb0362a36eacdae269f/heats_core/runner.py",
      "tests": "Passed: 211 repository tests, 0 failures after setting GIT_CEILING_DIRECTORIES to keep temporary non-repository fixtures from discovering the assignment checkout. First unisolated run had 5 fixture-location failures and 206 passes; no plugin source was changed. Supplemental full lifecycle/baseline/worktree/check/winner/apply run passed.",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 28,
      "rank": 1,
      "best": "It makes the user's agent accountable to real checks and turns independent worktree attempts into a tested, explicitly applied patch, demonstrated through a real Hermes child lifecycle.",
      "worst": "The safety policy handles commands and test-file tampering rather than providing an OS sandbox, and the author's small-model benchmark gain did not replicate (https://raw.githubusercontent.com/EconLearn/hermes-heats/2bcee10e9ca6489b7fd23bb0362a36eacdae269f/README.md).",
      "blocked": null
    },
    {
      "pr": 153606,
      "name": "habits",
      "validate": "Passed: catalog validate --install-deps --json exit 0; enable and doctor exit 0. Admission warning: private Hermes imports in approval/presence checks, skill-use counting and file safety may break across releases; no core override or dangerous-security-scan failure.",
      "inside": "Loaded: habit_run/habit_save (deferred discovery), seven hooks, /habits, CLI commands and two skills. tryit: a rules-only reflex ended successfully; missing habit returned a useful list of callable examples; an empty-node graph was refused; a traversal-style save name was refused. Second tryit plan: a synthetic read_file-to-end reflex returned READY; habit_save created its skill and replay page; calling the saved name succeeded structurally but returned Hermes's unchanged-file/dedup sentinel instead of READY; an absent start node was refused. This shows a repeat-use correctness gap for file-reading reflexes. No benchmark speedup, automatic promotion, live Jev service or personal-history learning is claimed as verified. Pinned code: https://raw.githubusercontent.com/emiwltz/hermes-habits/c2e32395e0e3461fe56c7d68b7bc993ee0954277/habits/plugin.py",
      "tests": "Installed-Hermes full suite: 4,896 passed, 14 failed, 28 skipped (4,938 collected). Five attachment tests encounter the plugin policy rejecting any hidden ancestor in the temporary fixture path; eight permission-switch cases expect chmod 600 to be flagged, whereas current Hermes detection treats it as safe; one secret fixture masks the agent parent module but leaves an already-imported secret_scope child available. A separate clean environment without Hermes ran all switch tests plus the attachment/secret probes: 339 passed, 1 attachment failure, supporting those environment differences. After installing jsonschema, the schema module passed all 43 tests, covering all 27 dependency skips; lsof benchmark check remains skipped. Eight tryit calls completed without a plugin crash but exposed the independent repeated-read/dedup defect. Hidden-path attachment policy: https://raw.githubusercontent.com/emiwltz/hermes-habits/c2e32395e0e3461fe56c7d68b7bc993ee0954277/habits/tools.py",
      "useful": 4,
      "reliable": 3,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 27,
      "rank": 5,
      "best": "It turns repeated work into versioned, guarded reflex graphs that can run several real tool steps without intermediate model turns and learn candidates from local history.",
      "worst": "A saved file-reading reflex returned a dedup sentinel instead of file content on reuse, and private Hermes APIs plus environment-sensitive attachment and permission behavior complicate compatibility (implementation: https://raw.githubusercontent.com/emiwltz/hermes-habits/c2e32395e0e3461fe56c7d68b7bc993ee0954277/habits/plugin.py).",
      "blocked": null
    }
  ]
}