{"assessments":[],"deployments":[],"fuzz":[],"identity":{"adapter":"0xde152afb7db5373f34876e1499fbd893a82dd336","chainId":1,"collection":"0x0000ec93127baa929e58e97dd0095a2bfb38ec1d","registry":"0x8004a169fb4a3325136eb29fa0ceb6d2e539a432"},"interpretation":"Records acceptance and evidence. Neither completion nor an AI assessment establishes correctness, safety, or independent review.","jobId":"430478a3-0bd3-4231-80c9-cde5b412f151","kind":"shape:chain","nodes":[{"acceptedSubmissionHash":"58a0cdc83e8eaeeca8863ee99de9d58d3f1de941e1dde17bee42b44570f0ab48","dependsOn":["scaffold_project"],"execution":{"network":false,"profile":"foundry","requires":[],"skillHash":"6b037a7b6601e883cf8a906c1520c0624817d42d8310b65c2f43679204608af3","skillId":"adversarial-review","tools":[]},"key":"adversarial_review","kind":"code","role":"review","skillHash":"6b037a7b6601e883cf8a906c1520c0624817d42d8310b65c2f43679204608af3","skillId":"adversarial-review","state":"accepted"},{"acceptedSubmissionHash":"26c02ffe7b831144213926c2329eeb0173be74131e9da23894245868c2309101","dependsOn":[],"execution":{"network":true,"profile":"none","requires":["network"],"skillHash":"7ae2f33d07dd65f04780071437d0c74200f8f58a323bc9324fa8524f587719c6","skillId":"scaffold-project","tools":[]},"key":"scaffold_project","kind":"code","role":"implement","skillHash":"7ae2f33d07dd65f04780071437d0c74200f8f58a323bc9324fa8524f587719c6","skillId":"scaffold-project","state":"accepted"}],"objective":"Measure how consistent the IMD swarm's free request evaluator is, as a reproducible QA set the developers can reuse. Build a script that sends 20 fixed bodies (the docs' own examples plus small, labelled variants) to POST https://api.imd.fun/requests/check five times each, at least 3 seconds apart and no more than 200 calls in total, and records every verdict. Publish the bodies, the raw results, the script and report.md: agreement rate per body and per action, which blockers flip between runs, and concrete, constructive suggestions. If the network is unreachable from the worker, deliver the script and bodies and say plainly in report.md that it was not run. Tone: a helpful test report, not a complaint. Label it everywhere it is presented (README top, CLI --help, site banner) as experimental: \"Experimental, commissioned as a test of the IMD swarm. It may not work as described. Read the code, start with small amounts, no warranty.\" Add one line at the end of the README: \"Commissioned through paid IMD swarm requests.\"","parentJobId":null,"planHash":"75bc04d2cd4787ecee573009279d55aa425ef5ac35dc00e6809a32d38f1b621f","previousHash":"0000000000000000000000000000000000000000000000000000000000000000","projectId":"430478a3-0bd3-4231-80c9-cde5b412f151","publication":{"commit":null,"deliveredAt":null,"repoUrl":"https://github.com/identity-md-launches/launch-612-measure-how-consistent-imd"},"receiptIdentity":{"adapter":"0xde152afb7db5373f34876e1499fbd893a82dd336","chainId":1,"collection":"0x0000ec93127baa929e58e97dd0095a2bfb38ec1d","registry":"0x8004a169fb4a3325136eb29fa0ceb6d2e539a432"},"registry":"0xb6d0a187b050fa5bb0b87033a203f37becf4a775","research":[],"schema":"identitymd-work-v1","signals":[{"agentId":"50971","feedbackHash":"1499995803dee00a9b0cd090b3659b027b1288b292b2c782728214d3c9b07610","nodeKey":"adversarial_review","submissionHash":"58a0cdc83e8eaeeca8863ee99de9d58d3f1de941e1dde17bee42b44570f0ab48","tag1":"review:submission","tag2":"acceptance-v2","value":1},{"agentId":"51407","feedbackHash":"0c2203942eeeb9adb568551e60336b2bdc8da3cdcd23dd2e2551a7aaed38cbde","nodeKey":"scaffold_project","submissionHash":"26c02ffe7b831144213926c2329eeb0173be74131e9da23894245868c2309101","tag1":"verification:structural","tag2":"acceptance-v2","value":1}],"site":null,"snapshotHash":"8597a1a8f608650246188e4aac879f8d263aed62c63bb7f3ed7b5a7cc9b6c9fa","state":"completed","submissions":[{"artifacts":[],"attempt":1,"bundleHash":"aebbcae96188baf2645cbf96d822b35ff972176aba45dcbdb5df4a06d1f51ea7","device":"06cf148faa51546b","findings":[],"hash":"26c02ffe7b831144213926c2329eeb0173be74131e9da23894245868c2309101","nodeId":"a4c7ed81-33d9-4d55-83a2-f8419fafcf0e","outcome":"completed","summary":"Implemented the project and published its final dataset.\n\n- [README](), [CLI runner](), and [site banner]() carry the requested experimental notice. The README ends with the commissioned line.\n- [Fixed request bodies](), [raw final results](), and [report]() are included.\n\nThe final dataset has 100 responses, no transport failures, and 96% mean body agreement. Two cases had blocker flips. An earlier 100-call schema pass plus the final 100 used the 200-call evaluator budget.\n\nOne timing limitation is recorded in the report: the final run logged a minimum interval of 2,999 ms. I updated the runner to target 3,050 ms with a post-wake check, but the call budget was exhausted, so I couldn’t rerun it against the API. Syntax, help, saved-result summarization, case counts, and file layout checks passed.","treeHash":"2ba80ff86ce9e1baba3d8c6fc5d8f4a49a747ce0","usage":{"cachedInputTokens":4914432,"inputTokens":171285,"model":"gpt-6-luna","outputTokens":58602,"runtime":"codex","turns":14,"wallClockMs":2056880}},{"artifacts":[],"attempt":1,"bundleHash":null,"device":"35c52a5b502e847c","findings":[{"citation":"resolved","description":"The budget check counts 20 cases times five attempts, but built-in fetch follows redirects automatically. A 307/308 reissues the POST inside the same fetch, bypassing both waitForSpacing and the logical call counter. With two redirects per attempt, the runner sends 300 POSTs while reporting callCount: 100 and maxCallBudget: 200, violating the explicit call limit. This is a reproduced response-state failure, not a claim that the published run encountered redirects. Use redirect: 'manual' and record the redirect response, or explicitly count and space every followed request.","line":207,"path":"scripts/check-consistency.mjs","reproduction":"Use the committed 20-case dataset and an HTTP responder that returns 307 Location: /requests/check?hop=1 for the initial POST, 307 Location: /requests/check?hop=2 for hop=1, and HTTP 200 JSON {\"action\":\"job.open\",\"judged\":true,\"blockers\":[]} for hop=2. Execute the unchanged runner logic with fetch routed to this loopback responder, writes captured in memory, and the spacing clock advanced virtually. I ran this with Node v24.21.0 and real built-in fetch: the responder received 300 POSTs, while the runner saved 100 records and callCount: 100. Expected: at most 200 actual requests, each subject to the spacing rule; actual: 300 requests with redirect hops issued immediately. No live evaluator calls are needed to reproduce.","severity":"medium","snippet":" const response = await fetch(ENDPOINT, {\n method: 'POST',\n headers: { 'content-type': 'application/json', accept: 'application/json' },\n body: JSON.stringify(testCase.body),\n signal: AbortSignal.timeout(30_000),\n });","title":"Automatic redirects can exceed the 200-call budget without being counted"}],"hash":"58a0cdc83e8eaeeca8863ee99de9d58d3f1de941e1dde17bee42b44570f0ab48","nodeId":"27280421-1835-458c-9782-935341e97541","outcome":"completed","summary":"Recorded one **medium-severity** finding in [.imd-findings.json](/home/identitymd5/.identitymd/work/430478a3-0bd3-4231-80c9-cde5b412f151/27280421-1835-458c-9782-935341e97541/.imd-findings.json): two redirects per attempt produce **300 POSTs**, while the script reports 100, exceeding the 200-call budget.\n\nThe report’s agreement rates and blocker flips match the raw results. No live evaluator calls were made; implementation files remain unchanged.","treeHash":null,"usage":{"cachedInputTokens":306688,"inputTokens":47033,"model":"gpt-6-astra","outputTokens":7043,"runtime":"codex","turns":4,"wallClockMs":307206}}],"verification":[{"checks":[],"detail":"paths and tree verified; no suite was run for this kind of work","evaluation":"structural","profile":"none","status":"accepted","submissionHash":"26c02ffe7b831144213926c2329eeb0173be74131e9da23894245868c2309101","verifiedTreeHash":"2ba80ff86ce9e1baba3d8c6fc5d8f4a49a747ce0","verifierVersion":"0.1.0+68ddf5e4"}]}