{ "key": "f-shortfall-cap", "approach": "Blind overlay on N. Each step the overlay rebuilds the post-action resting book (state.own_orders minus N cancels plus N placements), estimates p as the mid of the best combined bid/ask (N plus competitor), maps p to a latent score z under the documented jump-diffusion (sigma 0.02, jump intensity 0.0019, jump sd 0.4, jump mean 0 = midpoints of ParameterVariance bands) plus a 1-cent estimation noise, and computes, per side, the expected shortfall at 5% of the one-step arbitrage loss sum q*max(p_next-ask,0) (asks) / q*max(bid-p_next,0) (bids) by quadrature over the 5% tail of the two-component (diffusion / one-jump) predictive. Because each order loss is monotone in p_next, the tail set is shared and ES is linear in sizes; when ES exceeds the budget the nearest orders are cut first (new placements resized or dropped; resting orders cancelled). N cancels of ids removed by the overlay are filtered. No orders are added, so nothing needs hiding from N. Frozen: budget 5.0 dollars per side, spreads {4}. Checks: N saved by script, SHA-256 bf0d179f...ffae verified; N total edge on seed 310000 = 17.116002 as expected. Assembled file equals S1 on 5 seeds (310009, 310013, 310021, 310023 at spread 4; 410000). Off-mode equals N on 310000..310009. seconds_per_simulation measured for S1 with four single-process runs side by side (N alone: about 0.96 s).", "tried": [ { "variant": "budget 1.0, all spreads (tuning 310000..310199)", "n": 200, "mean_diff": -50.635709, "se": 7.240584 }, { "variant": "budget 5.0, all spreads (tuning 310000..310199)", "n": 200, "mean_diff": -22.892886, "se": 3.595016, "by_spread": { "1": { "runs": 47, "mean_diff": -21.950816, "se": 6.29425 }, "2": { "runs": 57, "mean_diff": -34.254934, "se": 7.796012 }, "3": { "runs": 53, "mean_diff": -27.166853, "se": 7.610222 }, "4": { "runs": 43, "mean_diff": -3.593361, "se": 5.299723 } } }, "Diagnostic on the 3 worst tuning seeds: the cap binds on most steps of the second half of high-edge runs, removing the near-touch size that earns retail edge; arb loss did not fall either (cancelled resting orders and resized placements appear to disrupt N). Not a coding error: off-mode equals N exactly." ], "next": "Drop absolute-ES caps. ES is proportional to size exactly as edge is, so a size cap mostly removes profit. If kept at all, the shortfall should be priced against expected retail edge per order (ES minus expected edge) and applied only to new placements, never by cancelling N resting orders; or act only on detected jump states.", "spreads_acted": [ 4 ], "off_equals_n": true, "code": "import math\nfrom statistics import NormalDist\nfrom orderbook_pm_challenge.types import Side, PlaceOrder, CancelOrder, CancelAll\n\n_ND = NormalDist()\n\n\ndef _phi(x):\n return 0.5 * math.erfc(-x / math.sqrt(2.0))\n\n\nclass Strategy(BaseStrategy):\n \"\"\"Expected-shortfall cap over _N: per side, the mean of the worst 5% one-step\n arbitrage losses of the resting book under a jump-diffusion predictive for p\n is kept under a budget by shrinking or dropping the nearest orders first.\"\"\"\n\n SIG = 0.02\n LAM = 0.0019\n JMU = 0.0\n JSD = 0.4\n ALPHA = 0.05\n M = 12\n\n def __init__(self, budget=5.0, spreads=(4,), record=None):\n self._n = _N()\n self._budget = budget\n self._spreads = set(spreads)\n self._on = None\n self._gone = set()\n self._rec = record\n\n def on_step(self, state):\n if self._on is None:\n b, a = state.competitor_best_bid_ticks, state.competitor_best_ask_ticks\n s = max(1, (a - b + 1) // 2) if a is not None and b is not None else 0\n self._on = s in self._spreads\n acts = list(self._n.on_step(state))\n if not self._on:\n return acts\n if self._gone:\n acts = [x for x in acts if not (isinstance(x, CancelOrder) and x.order_id in self._gone)]\n for x in acts:\n if isinstance(x, PlaceOrder) and x.client_order_id:\n self._gone.discard(x.client_order_id)\n return self._cap(state, acts)\n\n def _tails(self, p, h):\n # one-step predictive of p from score z with Gaussian + compound-jump increments\n sig, lam = self.SIG, self.LAM\n v = sig * sig + lam * (self.JSD ** 2 + self.JMU ** 2)\n s_now = math.sqrt(max(h, 1) * v)\n s_nxt = math.sqrt(max(h - 1, 1) * v)\n p = min(0.995, max(0.005, p))\n q = _ND.inv_cdf(p)\n z = q * s_now\n dens = math.exp(-0.5 * q * q) / math.sqrt(2 * math.pi) / s_now\n est = min(0.01 / max(dens, 1e-9), 3 * sig)\n s0 = math.sqrt(sig * sig + est * est)\n comps = ((1 - lam, 0.0, s0), (lam, self.JMU, math.sqrt(s0 * s0 + self.JSD ** 2)))\n\n def surv(x):\n return sum(w * (1 - _phi((x - m) / s)) for w, m, s in comps)\n\n lo, hi = 0.0, 20 * s0 + 5 * self.JSD\n for _ in range(40):\n mid = 0.5 * (lo + hi)\n if surv(mid) > self.ALPHA:\n lo = mid\n else:\n hi = mid\n xs = lo\n up, dn = [], []\n for w, m, s in comps:\n for sign, out in ((1, up), (-1, dn)):\n t = 1 - _phi((xs - sign * m) / s)\n if t <= 0:\n continue\n wt = w * t / self.ALPHA / self.M\n for j in range(self.M):\n u = 1 - t * (j + 0.5) / self.M\n x = sign * (m * sign + s * _ND.inv_cdf(min(u, 1 - 1e-15)))\n out.append((wt, _phi((z + x) / s_nxt)))\n return up, dn\n\n def _cap(self, state, acts):\n book = {o.order_id: o for o in state.own_orders}\n for x in acts:\n if isinstance(x, CancelAll):\n book = {}\n elif isinstance(x, CancelOrder):\n book.pop(x.order_id, None)\n new = [(i, x) for i, x in enumerate(acts) if isinstance(x, PlaceOrder)]\n bids = [o.price_ticks for o in book.values() if o.side == Side.BUY] + [x.price_ticks for _, x in new if x.side == Side.BUY]\n asks = [o.price_ticks for o in book.values() if o.side == Side.SELL] + [x.price_ticks for _, x in new if x.side == Side.SELL]\n cb, ca = state.competitor_best_bid_ticks, state.competitor_best_ask_ticks\n if cb is not None:\n bids.append(cb)\n if ca is not None:\n asks.append(ca)\n if not bids or not asks:\n return acts\n p = (max(bids) + min(asks)) / 200.0\n up, dn = self._tails(p, state.steps_remaining)\n extra = []\n drop = {}\n for side, tail in ((Side.SELL, up), (Side.BUY, dn)):\n items = []\n for o in book.values():\n if o.side == side:\n items.append((o.price_ticks, 0, o.order_id, o.remaining_quantity))\n for i, x in new:\n if x.side == side:\n items.append((x.price_ticks, 1, i, x.quantity))\n if not items:\n continue\n sg = 1 if side == Side.SELL else -1\n items.sort(key=lambda it: sg * it[0])\n unit = []\n tot = 0.0\n for it in items:\n pr = it[0] / 100.0\n e = sum(w * max(sg * (pp - pr), 0.0) for w, pp in tail)\n unit.append(e)\n tot += e * it[3]\n if self._rec is not None:\n self._rec.append(tot)\n excess = tot - self._budget\n for it, e in zip(items, unit):\n if excess <= 1e-12:\n break\n if e <= 0:\n continue\n q = it[3]\n cut = min(q, excess / e)\n keep = math.floor((q - cut) * 100 + 1e-9) / 100.0\n if keep < 0.01:\n keep = 0.0\n excess -= (q - keep) * e\n if it[1] == 1:\n drop[it[2]] = keep\n elif keep < q:\n extra.append(CancelOrder(it[2]))\n self._gone.add(it[2])\n excess += keep * e\n if not drop and not extra:\n return acts\n out = []\n for i, x in enumerate(acts):\n if i in drop:\n k = drop[i]\n if k <= 0:\n if x.client_order_id:\n self._gone.add(x.client_order_id)\n continue\n x = PlaceOrder(side=x.side, price_ticks=x.price_ticks, quantity=k, client_order_id=x.client_order_id)\n out.append(x)\n return out + extra\n", "code_sha256": "9950f6ddd03922021040cd9c4226f473e4037f6ea25c26cc5ee23c014cdd1bbd", "assembled_sha256": "b133143385083e27719171d849732d131f3aa9fb15d4ba36f1a02611972bef8d", "tuning": { "n": 200, "mean_diff": -0.772573, "se": 1.133779 }, "paired": { "first_seed": 410000, "last_seed": 410499, "n": 500, "complete": true, "incumbent_mean": 104.413753, "mean_diff": -4.004254, "se": 0.978793, "sum_d": -2002.127014, "sum_d2": 247047.070252, "retail_diff": -2.284456, "arb_diff": -1.719798, "failures": 0 }, "by_spread": { "1": { "runs": 115, "incumbent_mean": 28.495154, "mean_diff": 0.0, "se": 0.0, "sum_d": 0.0, "sum_d2": 0.0, "retail_diff": 0.0, "arb_diff": 0.0 }, "2": { "runs": 116, "incumbent_mean": 97.766262, "mean_diff": 0.0, "se": 0.0, "sum_d": 0.0, "sum_d2": 0.0, "retail_diff": 0.0, "arb_diff": 0.0 }, "3": { "runs": 138, "incumbent_mean": 128.405451, "mean_diff": 0.0, "se": 0.0, "sum_d": 0.0, "sum_d2": 0.0, "retail_diff": 0.0, "arb_diff": 0.0 }, "4": { "runs": 131, "incumbent_mean": 151.672482, "mean_diff": -15.283412, "se": 3.565079, "sum_d": -2002.127014, "sum_d2": 247047.070252, "retail_diff": -8.719298, "arb_diff": -6.564114 } }, "seconds_per_simulation": 2.224 }