# r6-fresh merge 02: f6-advance-opposite, f6-consecutive-retreats, f6-retreat-size-half Inputs (read as data, SHA-256 checked against the job manifest): `item-f6-advance-opposite` (b42b4e5e…cb5f), `item-f6-consecutive-retreats` (9b9bfd66…e44b), `item-f6-retreat-size-half` (c985e401…fcb15). All three hashes matched the given values. ## Summary - **No promotion.** None of the three overlays beats the incumbent MA (code_sha256 `5d1bd5272884f513b19f0e66a7c235f6c188a08e4f7787ac9346f1614dda3ed3`) by twice its standard error on the holdout seeds 450000–450499. All three items recommend keeping MA. The finalist seeds 900000–903999 were not run by any item (each says so in its notes), which is correct for screening items. - Scoreboard (paired holdout difference against MA, 500 seeds, candidate minus MA): | key | spreads acted | overall mean_diff | se | target-spread mean_diff | best target-spread t | verdict | |---|---|---|---|---|---|---| | f6-advance-opposite | [1, 2, 3, 4] | -0.7052 | 0.4648 | -0.7052 (se 0.4648, n 500) | 0.89 on s=4 | keep MA | | f6-consecutive-retreats | [1, 4] | -0.1055 | 0.1652 | -0.2144 (se 0.3359, n 246) | 0.37 on s=4 | keep MA | | f6-retreat-size-half | [1, 4] | 3.927e-05 | 2.016e-05 | 7.982e-05 (se 4.086e-05, n 246) | 1.96 on s=1 | keep MA | Target-spread figures recomputed by this merge from each item's `per_seed` list; for f6-consecutive-retreats it matches that item's own `target_spreads_1_4` field (-0.21443, se 0.33590). - **What was learned.** - *f6-advance-opposite* (pause opposite side for one callback after an exact one-tick rival advance, only orders within 1 tick): the worst of the three, -0.705 ± 0.465 overall. Losses are concentrated at s=1 (-2.66 ± 1.81), driven by two seeds (450450 -155.6 via arbitrage, 450036 -146.5 via lost retail). The s=3 tuning gain (+0.76 ± 0.22 for the `one_callback` variant) was not the selected variant, and the selected variant is flat at s=3 on holdout. s=4 is mildly positive (+0.32 ± 0.36), not significant. - *f6-consecutive-retreats* (withdraw a side for k=2 callbacks after two same-side rival retreats within 5 steps, s∈{1,4}): tuning looked good at s=1 (+3.92 ± 3.63, n=54) but did not replicate: holdout s=1 -0.54 ± 0.59, s=4 +0.12 ± 0.32. Its own `next` proposes restricting to s=1 with other thresholds; the holdout does not support that direction strongly. - *f6-retreat-size-half* (halve the retreating side's size at unchanged prices after an exact one-tick retreat, s∈{1,4}): practically inert. Holdout mean +0.0000393 ± 0.0000202 overall; d is exactly 0 on all 122 s=4 seeds and on 27 of 124 s=1 seeds; all three tuning variants (h=0.5, h=0.25, half+drop-nearest) produced byte-identical tuning statistics. The trigger almost never fires on a resting order that later fills, or the resize does not change fills. Its t≈1.96 at s=1 is on an effect of 0.00016 edge per run and is of no practical value. ## Checks done by this merge - `overlay_sha256` = SHA-256 of `overlay_code` (UTF-8): true for all three. - `code_sha256` = SHA-256 of MA's code + `\n\n` + overlay_code: true for all three. MA's code was fetched by script from https://api.imd.fun/artifacts/149883fee797ad95a91e535b7a74453587a3eb19ca87ab95f7d8585db160a8f4 (the source the items' brief names); its `code` field hashes to `5d1bd527…3ed3`, equal to each row's `base_sha256` and to the artifact's own `code_sha256` field. - From each row's `per_seed` (500 entries, seeds 450000–450499, each once): n, sum_d, mean_diff and se recomputed with SE = sqrt((Σd² − (Σd)²/n)/(n−1)/n) and matched the reported overall and by-spread values exactly. retail_diff + arb_diff = mean_diff to within 3e-11 everywhere. The tuning by-spread SEs also recompute. - The per-seed spread labels are identical across the three rows (s=1: 124, s=2: 143, s=3: 111, s=4: 122). Holdout base means agree across rows: overall 106.77992531608679; s1 52.178154051575, s2 88.61320954304685, s3 135.04550905697928, s4 157.85353348907952. Tuning base means agree where reported (s1 54.319234, s2 70.711318, s3 79.600487, s4 200.440531). - The off-target seeds are exactly 0 where they exist: 254 for each of consecutive-retreats and retreat-size-half, all with d == 0.0. - Check values (seeds 310000–310003) are 17.1160017503, 23.8203924577, 2.3761435866, 243.8721397061 in all three rows, and match the expected 6-dp values given in two of the rows. ## Inconsistencies and caveats 1. **off_target_zero for f6-advance-opposite is vacuous.** It acted on all spreads 1–4, so there are no off-target seeds. The item says this itself. Its overall figure is therefore its target figure. 2. **Tuning means use different denominators.** f6-advance-opposite's tuning mean_diff is over all 200 tuning seeds (all of them are on target). f6-consecutive-retreats (2.136) and f6-retreat-size-half (6.37e-6) report tuning mean_diff over their 97 target-spread seeds only (s1 54 + s4 43), as their by-spread sums show. Their holdout `overall` values, however, are over all 500 seeds, including 254 off-target zeros. Compare target-spread figures, not `overall`, across rows. The table above gives both. 3. **Field formats differ** (not errors in value): `failures` is `0` in consecutive-retreats and `[]` elsewhere; seed ranges appear as a string, a list or an object; consecutive-retreats' `check_values` lists actuals without expected values or pass flags, and its `worst_seeds` has no base/candidate totals. Python versions differ (3.13.5 for advance-opposite, 3.12.3 for the others), yet base results are identical. 4. **f6-retreat-size-half: flagged source.** The job reports that one source this item cites did not hold when it was read again. The item's JSON contains no URL at all, so this merge cannot tell which source or claim was meant. Its numeric results do not depend on an external citation, and they are internally consistent (hashes and statistics recompute). Treat its prose claims about simulator behaviour (for example, that quantities floor to 0.01 and that cancel/re-place keeps order IDs) as unverified by this merge. 5. **Tuning to holdout did not carry over** for all three items: advance-opposite selected the `one_callback_within_one_tick` variant (+0.080 ± 0.399), which went to -0.705 ± 0.465 on holdout; consecutive-retreats went from +2.14 ± 2.02 to -0.21 ± 0.34 on target. This fits the brief's warning that small seed counts cannot rank candidates. 6. None of the rows claims a promotion, and none ran the finalist seeds. ## Top (largest mean_diff/se on a target spread, holdout) With only three rows, all three are listed. Spreads whose SE is 0 (d identically 0) are excluded from the ratio. ### 1. f6-retreat-size-half: s=1, mean_diff/se = 1.9649 code_sha256: `b54db960e6562ce53319d6d80419d93cc01c42fb4362a8e468df42fd4d236363` overlay_sha256: `cfa1df206293d0bf711119784f358f38fd4eff2ec35ca41f5d608322213bd387` This overlay is appended to MA's code after two newline bytes (`\n\n`) to give `code_sha256`. The overlay_code below is exactly as given: ~~~~python _IncumbentStrategy = Strategy from orderbook_pm_challenge.types import PlaceOrder as _F6Place, CancelOrder as _F6Cancel, CancelAll as _F6CancelAll, Side as _F6Side from orderbook_pm_challenge.utils import quantize_down as _F6Quantize class Strategy(_IncumbentStrategy): def __init__(self): super().__init__() self._f6_previous = None self._f6_spread = None def on_step(self, state): actions = super().on_step(state) bid = state.competitor_best_bid_ticks ask = state.competitor_best_ask_ticks previous = self._f6_previous self._f6_previous = (bid, ask) if self._f6_spread is None: # Infer once from the rival's first quotes, never from later widths. self._f6_spread = max(1, (ask - bid + 1) // 2) if bid is not None and ask is not None else 0 if self._f6_spread not in (1, 4) or previous is None: return actions old_bid, old_ask = previous side = None old_quote = None if old_bid is not None and bid == old_bid - 1 and ask == old_ask: side, old_quote = _F6Side.BUY, old_bid elif old_ask is not None and ask == old_ask + 1 and bid == old_bid: side, old_quote = _F6Side.SELL, old_ask if side is None: return actions # Resolve MA's cancellations using only the callback's public orders. actions = list(actions) surviving = {o.order_id: o for o in state.own_orders} for action in actions: if isinstance(action, _F6CancelAll): surviving.clear() elif isinstance(action, _F6Cancel): surviving.pop(action.order_id, None) resting = [o for o in surviving.values() if o.side == side] placements = [(i, a) for i, a in enumerate(actions) if isinstance(a, _F6Place) and a.side == side] # The optional deletion selects one nearest order after MA's actions. nearest = None if False: choices = [(abs(o.price_ticks - old_quote), 0, j, ('rest', j)) for j, o in enumerate(resting)] choices += [(abs(a.price_ticks - old_quote), 1, i, ('new', i)) for i, a in placements] if choices: nearest = min(choices)[3] output = [_F6Cancel(o.order_id) for o in resting] for i, action in enumerate(actions): if isinstance(action, _F6Place) and action.side == side: q = _F6Quantize(float(action.quantity) * 0.5, 0.01) if q >= 0.01 and nearest != ('new', i): output.append(_F6Place(action.side, action.price_ticks, q, action.client_order_id)) else: output.append(action) for j, order in enumerate(resting): q = _F6Quantize(order.remaining_quantity * 0.5, 0.01) if q >= 0.01 and nearest != ('rest', j): output.append(_F6Place(order.side, order.price_ticks, q, order.order_id)) return output ~~~~ ### 2. f6-advance-opposite: s=4, mean_diff/se = 0.8948 code_sha256: `3499d71600b66c4252b306a9af5aeb1b4ff452e7cea7fafab41764a6a2dac0d6` overlay_sha256: `f6668f3d643f15cdb3b0f6fc248b1a81535ce575dc71fded3e9b43a658cc31d4` This overlay is appended to MA's code after two newline bytes (`\n\n`) to give `code_sha256`. The overlay_code below is exactly as given: ~~~~python _IncumbentStrategy = Strategy class Strategy(_IncumbentStrategy): """Pause opposite-side liquidity after an exact one-tick rival advance.""" def __init__(self): super().__init__() self._f6_advance_opposite_spread = None self._f6_advance_opposite_prev_bid = None self._f6_advance_opposite_prev_ask = None self._f6_advance_opposite_buy_until = -1 self._f6_advance_opposite_sell_until = -1 self._f6_advance_opposite_buy_reference = None self._f6_advance_opposite_sell_reference = None def on_step(self, state): actions = super().on_step(state) bid = state.competitor_best_bid_ticks ask = state.competitor_best_ask_ticks if self._f6_advance_opposite_spread is None: self._f6_advance_opposite_spread = ( max(1, (ask - bid + 1) // 2) if bid is not None and ask is not None else 0 ) if self._f6_advance_opposite_spread not in (1, 2, 3, 4): return actions previous_bid = self._f6_advance_opposite_prev_bid previous_ask = self._f6_advance_opposite_prev_ask if bid is not None and previous_bid is not None and bid == previous_bid + 1: self._f6_advance_opposite_sell_until = state.step + 0 self._f6_advance_opposite_sell_reference = bid if ask is not None and previous_ask is not None and ask == previous_ask - 1: self._f6_advance_opposite_buy_until = state.step + 0 self._f6_advance_opposite_buy_reference = ask self._f6_advance_opposite_prev_bid = bid self._f6_advance_opposite_prev_ask = ask block_buy = state.step <= self._f6_advance_opposite_buy_until block_sell = state.step <= self._f6_advance_opposite_sell_until if not block_buy and not block_sell: return actions from orderbook_pm_challenge.types import CancelAll, CancelOrder, PlaceOrder, Side def blocked(side, price): if side == Side.BUY and block_buy: return abs(price - self._f6_advance_opposite_buy_reference) <= 1 if side == Side.SELL and block_sell: return abs(price - self._f6_advance_opposite_sell_reference) <= 1 return False kept = [ action for action in actions if not (isinstance(action, PlaceOrder) and blocked(action.side, action.price_ticks)) ] # Preserve MA's cancels; its CancelAll already removes every resting order. if any(isinstance(action, CancelAll) for action in kept): return kept cancelled = { action.order_id for action in kept if isinstance(action, CancelOrder) } extra_cancels = [ CancelOrder(order.order_id) for order in state.own_orders if order.order_id not in cancelled and blocked(order.side, order.price_ticks) ] return extra_cancels + kept ~~~~ ### 3. f6-consecutive-retreats: s=4, mean_diff/se = 0.3669 code_sha256: `e0d0752730b68c0d8d30ae0c963b55cedd4d4afe90f2d186812848041e0a0ceb` overlay_sha256: `fd0cc980057029423edf6f93d352c3010149802818bc51e56a3e3780c5c093cd` This overlay is appended to MA's code after two newline bytes (`\n\n`) to give `code_sha256`. The overlay_code below is exactly as given: ~~~~python from orderbook_pm_challenge.types import CancelOrder, PlaceOrder, Side _IncumbentStrategy = Strategy class Strategy(_IncumbentStrategy): _K = 2 _WINDOW = 5 _SPREADS = (1, 4) def __init__(self): super().__init__() self._ov_s = None self._ov_bid = None self._ov_ask = None self._ov_dn = [] self._ov_up = [] self._ov_block_buy = -1 self._ov_block_sell = -1 def on_step(self, state): actions = list(super().on_step(state)) bid = state.competitor_best_bid_ticks ask = state.competitor_best_ask_ticks step = state.step if self._ov_s is None and bid is not None and ask is not None: self._ov_s = max(1, (ask - bid + 1) // 2) if bid is not None and self._ov_bid is not None and bid < self._ov_bid: self._ov_dn = [t for t in self._ov_dn if step - t <= self._WINDOW] + [step] if ask is not None and self._ov_ask is not None and ask > self._ov_ask: self._ov_up = [t for t in self._ov_up if step - t <= self._WINDOW] + [step] if bid is not None: self._ov_bid = bid if ask is not None: self._ov_ask = ask if self._ov_s not in self._SPREADS: return actions if len(self._ov_dn) >= 2: self._ov_block_buy = step + self._K - 1 self._ov_dn = [] if len(self._ov_up) >= 2: self._ov_block_sell = step + self._K - 1 self._ov_up = [] block_buy = step <= self._ov_block_buy block_sell = step <= self._ov_block_sell if not (block_buy or block_sell): return actions blocked = set() if block_buy: blocked.add(Side.BUY) if block_sell: blocked.add(Side.SELL) kept = [] cancelled = set() cancel_all = False for a in actions: if isinstance(a, PlaceOrder) and a.side in blocked: continue if isinstance(a, CancelOrder): cancelled.add(a.order_id) elif type(a).__name__ == 'CancelAll': cancel_all = True kept.append(a) if not cancel_all: for o in state.own_orders: if o.side in blocked and o.order_id not in cancelled: kept.append(CancelOrder(o.order_id)) return kept ~~~~ ## Merged JSON Numbers are copied from the items. `worst_seeds` per spread are the three most negative paired d values in each item's `per_seed` list. `tuning` is the selected variant. ```json { "rows": [ { "key": "f6-advance-opposite", "mechanism": "An exact +1 tick rival bid advance pauses own asks; an exact -1 tick rival ask advance pauses own bids.", "spreads_acted": [ 1, 2, 3, 4 ], "overlay_sha256": "f6668f3d643f15cdb3b0f6fc248b1a81535ce575dc71fded3e9b43a658cc31d4", "code_sha256": "3499d71600b66c4252b306a9af5aeb1b4ff452e7cea7fafab41764a6a2dac0d6", "base_sha256": "5d1bd5272884f513b19f0e66a7c235f6c188a08e4f7787ac9346f1614dda3ed3", "off_target_zero": true, "complete": true, "failures": [], "tuning": { "seeds": "340200..340399", "selected": "one_callback_within_one_tick", "mean_diff": 0.07960330221500138, "se": 0.399417673305197 }, "paired": { "seeds": "450000..450499", "n": 500, "mean_diff": -0.7051805508314004, "se": 0.4647761196368263, "retail_diff": -0.8121636984242003, "arb_diff": 0.1069831475916002, "base_mean": 106.77992531608679 }, "by_spread": { "1": { "n": 124, "sum_d": -329.4312505198, "sum_d2": 50959.011572161144, "mean_diff": -2.656703633224194, "se": 1.8121130291665557, "retail_diff": -3.294485536665322, "arb_diff": 0.6377819034209679, "base_mean": 52.178154051575, "worst_seeds": [ { "seed": 450450, "d": -155.61885714240003 }, { "seed": 450036, "d": -146.53290124859996 }, { "seed": 450465, "d": -52.161229249800044 } ] }, "2": { "n": 143, "sum_d": -59.00264927070003, "sum_d2": 1179.1423834921807, "mean_diff": -0.4126059389559443, "se": 0.23847383404831748, "retail_diff": -0.16907309433916107, "arb_diff": -0.24353284461818087, "base_mean": 88.61320954304685, "worst_seeds": [ { "seed": 450273, "d": -32.190272034299994 }, { "seed": 450272, "d": -4.924160814500006 }, { "seed": 450181, "d": -4.4566300324 } ] }, "3": { "n": 111, "sum_d": -3.74023664810003, "sum_d2": 52.94528282740487, "mean_diff": -0.03369582565855883, "se": 0.06577158279087132, "retail_diff": -0.04048538666306418, "arb_diff": 0.006789561021621355, "base_mean": 135.04550905697928, "worst_seeds": [ { "seed": 450324, "d": -2.242988287899948 }, { "seed": 450125, "d": -1.504796934799998 }, { "seed": 450307, "d": -1.4787113825999967 } ] }, "4": { "n": 122, "sum_d": 39.583861022899825, "sum_d2": 1953.7424916287348, "mean_diff": 0.3244578772368838, "se": 0.36260090991219374, "retail_diff": 0.2549646536434419, "arb_diff": 0.06949322359508185, "base_mean": 157.85353348907952, "worst_seeds": [ { "seed": 450475, "d": -3.473707485099993 }, { "seed": 450135, "d": -1.0390508783999906 }, { "seed": 450408, "d": -0.7085166366000237 } ] } } }, { "key": "f6-consecutive-retreats", "mechanism": "two rival retreats same side within 5 steps: cancel that side and quote nothing there for k callbacks", "spreads_acted": [ 1, 4 ], "overlay_sha256": "fd0cc980057029423edf6f93d352c3010149802818bc51e56a3e3780c5c093cd", "code_sha256": "e0d0752730b68c0d8d30ae0c963b55cedd4d4afe90f2d186812848041e0a0ceb", "base_sha256": "5d1bd5272884f513b19f0e66a7c235f6c188a08e4f7787ac9346f1614dda3ed3", "off_target_zero": true, "complete": true, "failures": 0, "tuning": { "seeds": "340200..340399", "selected": "k=2", "mean_diff": 2.1359969050855674, "se": 2.0247500796395244 }, "paired": { "seeds": "450000..450499", "n": 500, "mean_diff": -0.10549961322720006, "se": 0.1651598569458485, "retail_diff": -0.18312384201380028, "arb_diff": 0.07762422878180006, "base_mean": 106.77992531608679 }, "by_spread": { "1": { "n": 124, "sum_d": -66.9172512315001, "sum_d2": 5330.799013808594, "mean_diff": -0.5396552518669363, "se": 0.5891918596282573, "retail_diff": -0.7459811702048385, "arb_diff": 0.20632591834112904, "base_mean": 52.178154051575, "worst_seeds": [ { "seed": 450036, "d": -49.08000692529993 }, { "seed": 450202, "d": -37.538154083500004 }, { "seed": 450322, "d": -8.927738552400001 } ] }, "2": { "n": 143, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "retail_diff": 0.0, "arb_diff": 0.0, "base_mean": 88.61320954304685, "worst_seeds": [] }, "3": { "n": 111, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "retail_diff": 0.0, "arb_diff": 0.0, "base_mean": 135.04550905697928, "worst_seeds": [] }, "4": { "n": 122, "sum_d": 14.167444617900074, "sum_d2": 1480.571767807049, "mean_diff": 0.11612659522868914, "se": 0.3165196686306058, "retail_diff": 0.0077028204795068595, "arb_diff": 0.1084237747262297, "base_mean": 157.85353348907952, "worst_seeds": [ { "seed": 450019, "d": -20.35826098680002 }, { "seed": 450159, "d": -8.916855684600023 }, { "seed": 450114, "d": -3.552341952800006 } ] } } }, { "key": "f6-retreat-size-half", "mechanism": "Exactly one-tick rival retreat on one side with the opposite quote unchanged; cancel and resize that side at unchanged prices.", "spreads_acted": [ 1, 4 ], "overlay_sha256": "cfa1df206293d0bf711119784f358f38fd4eff2ec35ca41f5d608322213bd387", "code_sha256": "b54db960e6562ce53319d6d80419d93cc01c42fb4362a8e468df42fd4d236363", "base_sha256": "5d1bd5272884f513b19f0e66a7c235f6c188a08e4f7787ac9346f1614dda3ed3", "off_target_zero": true, "complete": true, "failures": [], "tuning": { "seeds": "340200..340399", "selected": "half", "mean_diff": 6.368497938683923e-06, "se": 2.7940363736092244e-05 }, "paired": { "seeds": "450000..450499", "n": 500, "mean_diff": 3.927036319956024e-05, "se": 2.0159203436869843e-05, "retail_diff": -5.219850799970546e-06, "arb_diff": 4.449021440007483e-05, "base_mean": 106.77992531608679 }, "by_spread": { "1": { "n": 124, "sum_d": 0.01963518159978012, "sum_d2": 0.00010216625477358436, "mean_diff": 0.0001583482387079042, "se": 8.058965843611418e-05, "retail_diff": -2.1047785483752203e-05, "arb_diff": 0.00017939602580675337, "base_mean": 52.178154051575, "worst_seeds": [ { "seed": 450052, "d": -0.0028965400000000585 }, { "seed": 450389, "d": -0.0008367840000000071 }, { "seed": 450076, "d": -0.0005824972999981526 } ] }, "2": { "n": 143, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "retail_diff": 0.0, "arb_diff": 0.0, "base_mean": 88.61320954304685, "worst_seeds": [] }, "3": { "n": 111, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "retail_diff": 0.0, "arb_diff": 0.0, "base_mean": 135.04550905697928, "worst_seeds": [] }, "4": { "n": 122, "sum_d": 0.0, "sum_d2": 0.0, "mean_diff": 0.0, "se": 0.0, "retail_diff": 0.0, "arb_diff": 0.0, "base_mean": 157.85353348907952, "worst_seeds": [] } } } ], "top": [ { "rank": 1, "key": "f6-retreat-size-half", "best_target_spread": 1, "mean_diff_over_se": 1.9648704533650747, "code_sha256": "b54db960e6562ce53319d6d80419d93cc01c42fb4362a8e468df42fd4d236363", "overlay_sha256": "cfa1df206293d0bf711119784f358f38fd4eff2ec35ca41f5d608322213bd387" }, { "rank": 2, "key": "f6-advance-opposite", "best_target_spread": 4, "mean_diff_over_se": 0.8948071236651156, "code_sha256": "3499d71600b66c4252b306a9af5aeb1b4ff452e7cea7fafab41764a6a2dac0d6", "overlay_sha256": "f6668f3d643f15cdb3b0f6fc248b1a81535ce575dc71fded3e9b43a658cc31d4" }, { "rank": 3, "key": "f6-consecutive-retreats", "best_target_spread": 4, "mean_diff_over_se": 0.3668858738892926, "code_sha256": "e0d0752730b68c0d8d30ae0c963b55cedd4d4afe90f2d186812848041e0a0ceb", "overlay_sha256": "fd0cc980057029423edf6f93d352c3010149802818bc51e56a3e3780c5c093cd" } ] } ``` ## Sources - Item results: `.imd/reads/artifacts/item-f6-advance-opposite`, `item-f6-consecutive-retreats`, `item-f6-retreat-size-half` (artifacts/goal/batches/r6-fresh/items/*/result.json). - MA (incumbent) artifact: https://api.imd.fun/artifacts/149883fee797ad95a91e535b7a74453587a3eb19ca87ab95f7d8585db160a8f4. It was used only to hash-check `base_sha256` and `code_sha256`. - Simulator: https://github.com/danrobinson/prediction-market-challenge at b244530ee2a6247b5f930830eeca9dd950da0e7e. The items ran it; this merge did not rerun it.