{
  "batch": "deep-20",
  "setup": "Installed real Hermes from source at fdde27e7dae174cb4ba842589e92508852b7dd9f using uv sync --locked --python 3.14; Python 3.14.10. The CLI listed install, validate and doctor. Followed https://gist.githubusercontent.com/gtmcknight/594c47336546c8763e641b46cf9af4ec/raw/RUN.md and ran its tryit.py with Nous's scripted FakeLLMServer: empty plans first, then 3-5 planned calls, including bad inputs. All plugin checkouts used field.json's exact SHA and subdirectory. All work and test profiles were confined to test/scratch. Installed pytest, pytest-asyncio and PyYAML into the scratch Hermes environment. No paid model, account or device credentials used. Samskara additionally needed memory.provider=samskara and SAMSKARA_ENDPOINT=embedded://seed/demo.ttl in the test run; only the scratch runner configuration changed, not plugin code. Results distinguish locally observed behavior from README claims; Desktop rendering, real phone audio and optional external CAD/video programs were not independently exercised. Rankings are within these six entries only, prioritizing lasting installation value when close. At tied totals I prefer the recurring coding assistance from Locus over the specialized creation workflow from Construct, and working activation in Vellum over failed ordinary activation in Workgraph. Catalog policy read at https://raw.githubusercontent.com/NousResearch/hermes-agent/fdde27e7dae174cb4ba842589e92508852b7dd9f/plugin-catalog/README.md; pinned repository/code URLs below are also in sources.json. Test counts and runtime results are this seat's own observations, not claims copied from entrants.",
  "plugins": [
    {
      "pr": 150453,
      "name": "workgraph",
      "validate": "Passed: plugins validate --install-deps --json exited 0, all 15 admission checks passed, security scan safe and zero warnings. Doctor --ci exited 0, with 1 tool and 3 hooks. Normal plugins enable failed twice: uv lock could not find the staged editable workgraph-agent distribution. Admission validation passing therefore did not mean successful activation.",
      "inside": "The stock empty-plan and five-call runs could import/register the plugin for doctor but failed normal enablement; its workgraph tool was consequently unavailable in the chat. For diagnosis only, explicitly enabled it in a scratch profile configuration, without changing plugin source: all five calls then returned on a fresh scratch Git fixture (status, open attempt, checkpoint, history, bad version id). The checkpoint selected a revision, history listed one saved version, and the bad id returned NOT_FOUND. Code registers tool_request/tool_execution middleware for managed file/terminal routing in addition to its three hooks. This manual profile result does not erase the normal activation failure. No real browser gallery/protected-flow recording was exercised. Pinned adapter: https://raw.githubusercontent.com/aliiqbal24/workgraph/37c7280a514d3fa23e7193f44b5123d1f9c90163/workgraph/hermes_adapter.py; tool contract: https://raw.githubusercontent.com/aliiqbal24/workgraph/37c7280a514d3fa23e7193f44b5123d1f9c90163/workgraph/api.py",
      "tests": "Failed: python -m unittest discover -s tests -v ran 69 tests in 1126.583 seconds: 56 passed, 7 failures, 6 errors. Errors were failed-candidate acceptance paths; several failure assertions depended on sandbox execution that returned a Bubblewrap initialization error in this run, and one concurrency check exceeded its 0.25-second bound. An isolated acceptance-test repeat also errored, and a separate minimal fixture check confirmed a Bubblewrap error rather than an application assertion. These sandbox-dependent results are not proof of an upstream source regression; normal Hermes activation nevertheless independently failed.",
      "rank": 6,
      "best": "Its saved product-state tree, data-aware branching and automatic managed-edit checkpoints address real agent work recovery rather than merely describing a Git workflow.",
      "worst": "The ordinary enable command failed, so even the successful manually enabled versioning path is not a dependable day-one installation; its documented Linux, Bubblewrap and browser prerequisites add setup work (https://raw.githubusercontent.com/aliiqbal24/workgraph/37c7280a514d3fa23e7193f44b5123d1f9c90163/README.md).",
      "useful": 4,
      "reliable": 2,
      "polish": 4,
      "creative": 5,
      "beyond": 5,
      "total": 25,
      "blocked": null
    },
    {
      "pr": 152533,
      "name": "vellum",
      "validate": "Passed: plugins validate --install-deps exited 0; doctor --ci exited 0, with 4 tools and 5 hooks; the catalog accepted the Desktop surface.",
      "inside": "Loaded vellum_draw, vellum_look, vellum_clear and vellum_read plus 5 turn/stream hooks and a paper-canvas skill. All four planned calls returned cleanly in real Hermes: read with no open paper, draw on pa-test, malformed canvas ../bad, and clear. The hooks refused them outside an authenticated paper turn rather than waiting for Desktop. This verifies guarded behavior and registration, not successful drawing or speech. No running Desktop paper/browser was available in the prescribed headless runner, so in-page rendering and voice synchronization were assessed from code/tests. Turn gating: https://raw.githubusercontent.com/0xsaltylollipop/vellum/286b44d587500bd4792decdc73370791262aec0b/paper_anim/turns.py",
      "tests": "Failed: python -m pytest tests -q, 72 passed and 1 failed. test_a_long_reply_costs_the_same_per_piece_of_text exceeded its 2-second bound (about 4.96 seconds) during the first suite run. An isolated rerun of that test failed with RuntimeError: can't start new thread in _Voice.feed at threading.Timer(...).start(). After the other plugin suites finished, a final full-suite rerun again had 72 passes and 1 failure in 1.98 seconds, with the same timer-thread RuntimeError. This is a repeatable stress-test issue, not a crash of one of the four guarded Hermes tool calls. Test source: https://raw.githubusercontent.com/0xsaltylollipop/vellum/286b44d587500bd4792decdc73370791262aec0b/tests/test_vellum.py",
      "rank": 5,
      "best": "A canvas embedded under the highlighted paper passage, synchronized through Hermes stream hooks, is a distinctive extension a prompt cannot provide.",
      "worst": "The long-stream regression test failed repeatedly, including timer-thread exhaustion, and the live Desktop drawing/voice experience was not independently exercised (https://raw.githubusercontent.com/0xsaltylollipop/vellum/286b44d587500bd4792decdc73370791262aec0b/paper_anim/turns.py; https://raw.githubusercontent.com/0xsaltylollipop/vellum/286b44d587500bd4792decdc73370791262aec0b/README.md).",
      "useful": 3,
      "reliable": 3,
      "polish": 4,
      "creative": 5,
      "beyond": 5,
      "total": 25,
      "blocked": null
    },
    {
      "pr": 149823,
      "name": "samskara",
      "validate": "Passed: plugins validate --install-deps --json exited 0, all 15 checks passed, security scan safe, zero warnings; doctor --ci exited 0 and registered the exclusive memory provider.",
      "inside": "Loaded. Empty default-profile run correctly added no model tools until the external memory provider was selected. With memory.provider=samskara and the bundled embedded endpoint, Hermes offered samskara_search, samskara_query, samskara_write, samskara_conflicts and samskara_status. All four planned calls returned: healthy status; PostgreSQL search with 3 entities and 15 facts; DELETE query refused as read-only; one explicit fact queued with 1 triple pending reindex. The embedded status could not report entity count. No HTTP SPARQL server was tested. Pinned registration and tool handlers: https://raw.githubusercontent.com/SebaWag/hermes-samskara/d807e875da3176301db9955e0352c9c8adb6cb01/__init__.py",
      "tests": "Passed: python -m pytest tests -q, 150 passed in 4.61 seconds, no failures or skips; includes embedded store, facts, conflicts, extraction, failure modes and trust boundaries. Live graph integration was not run.",
      "rank": 4,
      "best": "A real memory-provider integration gives Hermes provenance-bearing relational recall and explicit contradictory facts, rather than just another search prompt.",
      "worst": "Writes require a separate reindex before graph recall sees them, and the embedded backend supports only a narrow set of query shapes (https://raw.githubusercontent.com/SebaWag/hermes-samskara/d807e875da3176301db9955e0352c9c8adb6cb01/README.md; https://raw.githubusercontent.com/SebaWag/hermes-samskara/d807e875da3176301db9955e0352c9c8adb6cb01/samskara_core/embedded.py).",
      "useful": 4,
      "reliable": 4,
      "polish": 4,
      "creative": 4,
      "beyond": 5,
      "total": 26,
      "blocked": null
    },
    {
      "pr": 153357,
      "name": "twring",
      "validate": "Passed: plugins validate --install-deps exited 0; doctor --ci exited 0 and registered the platform and CLI; no model tools are expected from this platform plugin.",
      "inside": "Loaded as the twring platform and hermes twring CLI, adding no model tool. Three planned calls through real Hermes terminal returned: twring status clearly reported not linked (exit 1); link --server ftp://invalid.example --name review-fixture rejected the scheme immediately (exit 1, no link request); twring --help listed link, status and unlink (exit 0). Platform tests separately ran the actual Hermes adapter against a local fake protocol peer. Real Twring account linking, phone audio and notification ringing were not tested. Pinned implementation: https://raw.githubusercontent.com/saasagelabs/twring/5a7083e3c9d1cd656f1fd85da771d6eee3d2267a/integrations/hermes/adapter.py",
      "tests": "Passed: python -m pytest -p pytest_asyncio.plugin tests -q in integrations/hermes, 84 passed in 12.41 seconds, no skips or failures. Covers protocol, link/CLI, and real Hermes adapter with a fake WebSocket peer, including streaming, cancellation and approval handling. The wider mobile/server monorepo suite was not run. Test source: https://raw.githubusercontent.com/saasagelabs/twring/5a7083e3c9d1cd656f1fd85da771d6eee3d2267a/integrations/hermes/tests/test_adapter.py",
      "rank": 3,
      "best": "It makes the user's existing Hermes reachable by live call and carries human approvals and scheduled-job callbacks through a native messaging platform.",
      "worst": "The complete experience requires a Twring account, phone app and running gateway/server, whose real audio and ringing path remains unverified here (https://raw.githubusercontent.com/saasagelabs/twring/5a7083e3c9d1cd656f1fd85da771d6eee3d2267a/integrations/hermes/README.md).",
      "useful": 5,
      "reliable": 4,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 28,
      "blocked": null
    },
    {
      "pr": 153687,
      "name": "locus-codegraph",
      "validate": "Passed: plugins validate --install-deps exited 0, doctor --ci exited 0, 5 tools and 3 hooks registered; catalog probe and Desktop surface checks passed.",
      "inside": "Loaded 5 tools (locus_map, locus_rehearse, locus_symbol, locus_impact, locus_hotspots), 3 lifecycle/edit hooks, context references, commands and UI integrations. Five planned calls on a two-file Python fixture: map found 2 files/2 symbols/1 edge; rehearsal of audience->scope named checkout.py:3 and supplied a caller patch; Hermes patch changed the definition; hotspots reported 1 open drift alert; empty symbol name returned a clear error. The stock runner launches its agent in the scratch profile, so its initial edit hook did not analyze the explicit tool workspace. Repeated with the child process launched in the fixture and AIAgent cwd set there, matching normal project use: the patch result itself carried the Signature Drift Alert, binding reason and caller repair patch. No plugin code changed. Pinned implementation: https://raw.githubusercontent.com/atikur/hermes-locus/e083a7c6df70f4f4d54a5247890deb14015d689e/__init__.py; rehearsal: https://raw.githubusercontent.com/atikur/hermes-locus/e083a7c6df70f4f4d54a5247890deb14015d689e/locus_core/rehearsal.py",
      "tests": "Passed using the documented runner: python -m unittest discover -s tests -v, 146 tests in 32.450 seconds, no skips or failures; node --test tests/ui.test.cjs, 10 passed. An initial pytest attempt produced 153 setup/import errors because it imported the relative-import plugin root as a bare __init__ module; switching to the README unittest command resolved them. Native UI rendering was not tested.",
      "rank": 1,
      "best": "Rehearsal and immediate caller-specific edit feedback measurably improve Hermes's own coding loop without another model call or hosted index.",
      "worst": "Its checks depend on statically proven Python bindings and the active project directory, so dynamic/decorated calls remain outside the guarantee (https://raw.githubusercontent.com/atikur/hermes-locus/e083a7c6df70f4f4d54a5247890deb14015d689e/README.md).",
      "useful": 5,
      "reliable": 5,
      "polish": 5,
      "creative": 4,
      "beyond": 5,
      "total": 29,
      "blocked": null
    },
    {
      "pr": 150346,
      "name": "construct",
      "validate": "Passed: plugins validate --install-deps exited 0, zero warnings in its report; doctor --ci exited 0, registering 12 tools and 1 hook; security and Desktop surface checks passed.",
      "inside": "Loaded 12 construct_* tools, the approval hook, /construct command, a skill and Desktop/backend integration. All five planned calls ran inside real Hermes: construct_start created a two-part object project; construct_check returned ok with 0 errors/0 warnings; construct_build(thumbnail=false) wrote the HTML/WebGL artifact; construct_export(format=glb) succeeded; construct_edit on nonexistent part not-a-part returned written=false and a specific error. Independently opened the generated GLB: glTF version 2, declared length matched 41,208 bytes, 5 nodes, 2 meshes, named body and handle. HTML artifact and JSON scene were present and readable after the rejected edit. No rendered Desktop/browser session or physical fabrication was tested. Tool implementation: https://raw.githubusercontent.com/rtmandreyev/construct/24a1bc838076c94728376850d461b383d003caa9/hermes/construct/tools.py",
      "tests": "Passed: python -m pytest construct_hologram -q, 399 passed, 46 skipped, 78 subtests passed, 1 dependency deprecation warning, no failures, in 1045.96 seconds. This includes the Hermes packaging/tool/backend/approval tests. Optional browser, CAD, Blender and NetLogo-dependent paths account for skipped coverage; the successful headless build and GLB export were additionally checked inside real Hermes.",
      "rank": 2,
      "best": "One checked named-part scene becomes both a self-contained animated artifact and a read-back-verified model export, with a substantial Hermes Desktop and tool integration.",
      "worst": "Physical print quality and native Desktop presentation remain unverified here, while CAD, film and live NetLogo paths require optional external programs (https://raw.githubusercontent.com/rtmandreyev/construct/24a1bc838076c94728376850d461b383d003caa9/hermes/construct/README.md).",
      "useful": 4,
      "reliable": 5,
      "polish": 5,
      "creative": 5,
      "beyond": 5,
      "total": 29,
      "blocked": null
    }
  ]
}