# arb-3: router likelihood per price hypothesis

## Problem (quoted)

> **Arb or retail.** Worth 2.7. Tried, no gain:

Taylor identifies uncertainty about whether a step's first own trade is arbitrage or retail as an opportunity worth about 2.7. Tried approaches did not help: excluding the overlapping “quote zone” cost 0.6–2.4 under perfect information while retaining 13–15% classification error; pin-seeking with “tighter quotes” beyond kappa 1e-5 cost more than it earned; and a “quote-aware fill likelihood” also failed. A feature-table classifier had 18.8% error versus the lattice filter's 16.2%. Of the three untested proposals, the third is:

> router likelihood per price hypothesis

Source: [Taylor (gtmcknight), pinned solved-game README](https://raw.githubusercontent.com/gtmcknight/amm-sim/4812a6382311624e1506aca131ca35c378f930d9/research/solved/README.md), Open 3. This is the third untested idea in the order written. The quoted 2.7 is Taylor's estimate, not an outcome of this experiment.

## Idea

For each candidate fair price in the current no-arbitrage interval, first clip the estimated rival spot into that price's 30-bps band. Invert the exact constant-product router to recover the total retail order needed to produce the observed own fill. Evaluate the lognormal order-size density, including the Jacobian from log spot movement to order size, then average it over the Gaussian price hypotheses. Use this marginal likelihood in the existing arb/retail likelihood ratio. Two variants also use the likelihood-weighted retail posterior mean and variance.

This implements the selected idea in the Gaussian policy, not a new lattice policy. Rival spot is still estimated: the inverse is exact conditional on that estimate and the price hypothesis, not an exact Bayesian model of the hidden rival state. The router equations and order law come from [replica2.py](https://raw.githubusercontent.com/gtmcknight/amm-sim/4812a6382311624e1506aca131ca35c378f930d9/research/solved/replica2.py); the retained filter, mixture, phase rules and lookup come from [real.py](https://raw.githubusercontent.com/gtmcknight/amm-sim/4812a6382311624e1506aca131ca35c378f930d9/research/solved/real.py).

For either side, let C be the input reserve, g our fee multiplier, G=0.997, r=sqrt(g/G), and u=(C_after-C_before)/g. The implied total input is u+max(0,C_after/r-C_rival)/G. Its derivative with respect to absolute log spot movement is C_after/2 times (1/g plus 1/(rG) when the rival fills). For X input, convert input and derivative into Y using each candidate price. Nine midpoint nodes cover the no-arb interval intersected with mean ±6 standard deviations; normalized Gaussian weights are multiplied by the reference truncation mass Z. The likelihood multiplier rate/2 is an effective first-retail intensity, not a learned run parameter.

## Change (complete unified diff)

Credit: original policy, tables and replica by Taylor (gtmcknight). First run `cp real.py router_real.py` in `research/solved`, then apply this patch with `patch -p1` at commit `4812a6382311624e1506aca131ca35c378f930d9`; it modifies only the sibling, leaving `real.py` unchanged. All variants use unchanged `pol_mid.json`, `soft=1`, nine nodes and the original arb likelihood. `router=0` retains the original scoring path. Only the diagnostic counters read ground truth; the new decision path never reads `arb`, `p`, `s2`, `j` or run parameters passed to `init`.

```diff
--- a/router_real.py
+++ b/router_real.py
@@ -59,6 +59,7 @@
         self.fb = self.fa = 0.003
         self.sp = 100.0               # own spot after our last callback
         # diagnostics
+        self.tp = self.fp = self.tn = self.fn = self.calls = 0
         self.nmis = 0; self.ncls = 0; self.se = 0.0; self.nse = 0; self.se2 = 0.0
     def init(self, *a):
         fb, fa = self._fees(0.0, 0.0, 0)
@@ -97,6 +98,44 @@
         fa = 1.0 - math.exp(-(a - w) * 1e-4) if a > w else 0.0
         fb = 1.0 - math.exp(-(b + w) * 1e-4) if b > -w else 0.0
         return fb, fa
+    def router_likelihood(self, is_buy, s, m, v, lo, hi, Z):
+        # Exact inverse of replica2's router, conditional on a fair-price hypothesis.
+        # Rival's unobserved starting spot is estimated, then arbed into its band.
+        sd = math.sqrt(v)
+        left = max(lo, m - 6.0 * sd)
+        right = min(hi, m + 6.0 * sd)
+        if right <= left or Z <= 1e-12:
+            return 0.0
+        n = int(self.o.get('nodes', 9))
+        step = (right - left) / n
+        g = 1.0 - (self.fb if is_buy else self.fa)
+        r = math.sqrt(g / G2)
+        before = math.sqrt(R.K / self.sp) if is_buy else math.sqrt(R.K * self.sp)
+        after = math.sqrt(R.K / s) if is_buy else math.sqrt(R.K * s)
+        own = abs(after - before) / g
+        total = mass = first = second = 0.0
+        for k in range(n):
+            lp = left + (k + 0.5) * step
+            price = math.exp(lp)
+            rival = min(max(self.s2h, price * G2), price / G2)
+            other = math.sqrt(R.K / rival) if is_buy else math.sqrt(R.K * rival)
+            spill = max(0.0, after / r - other) / G2
+            scale = price if is_buy else 1.0
+            order = (own + spill) * scale
+            jac = 0.5 * after * (1.0 / g + (1.0 / (r * G2) if spill > 0 else 0.0)) * scale
+            density = phi((math.log(max(order, 1e-12)) - (math.log(20.0) - 0.72)) / 1.2) / (max(order, 1e-12) * 1.2)
+            weight = phi((lp - m) / sd)
+            likelihood = weight * density * jac
+            total += likelihood
+            first += likelihood * (lp - m)
+            second += likelihood * (lp - m) ** 2
+            mass += weight
+        if total > 0.0 and self.o.get('update', 0):
+            shift = first / total
+            self.router_mt = m + shift
+            self.router_vt = max(0.0, second / total - shift * shift)
+        return Z * 0.5 * self.o.get('rate', 0.8) * total / mass if mass else 0.0
+
     def on_trade(self, is_buy, ax, ay, t, x, y, arb, p, s2, j):
         s = y / x
         dt = t - self.tlast
@@ -137,6 +176,12 @@
                 l_ret = Z * 0.5 * H0 * jac * phi((math.log(D) - LD_FULL) / 1.2) / (D * 1.2)
             else:
                 l_ret = Z * RHO * phi((math.log(max(d, 1e-9)) - LD_M) / LD_S) / (max(d, 1e-9) * LD_S)
+            if o.get('router', 0):
+                lr = self.router_likelihood(is_buy, s, m, v, lo, hi, Z)
+                if o.get('update', 0) and lr > 0.0:
+                    mt, vt = self.router_mt, self.router_vt
+                blend = o.get('blend', 1.0)
+                l_ret = (1.0 - blend) * l_ret + blend * lr
             pi = l_arb / (l_arb + l_ret) if (l_arb + l_ret) > 0 else 0.0
             if o.get('oraclearb', 0): pi = 1.0 if arb else 0.0
             is_arb = pi > 0.5
@@ -167,6 +212,11 @@
             phs = 1 if self.cnt <= 1 else 2
         else:
             phs = 0
+        self.calls += 1
+        if is_arb and arb: self.tp += 1
+        elif is_arb: self.fp += 1
+        elif arb: self.fn += 1
+        else: self.tn += 1
         err = (self.m - math.log(p)) * 1e4
         self.se += err * err; self.nse += 1
         if not is_arb: self.se2 += err * err
```

Selected options: `plain:soft=1,router=1,rate=0.55,update=1` in `router_real.py` (label “plain” is arbitrary, as in the original CLI).

## Choose-set results (every setting tried)

Computed here, paired replica edge difference = candidate minus unchanged `real.py pol_mid.json plain:soft=1`. Each seed uses the replica's default 10,000 steps and default randomized run configuration. Each row has n=1,024 and exactly seeds 60000..61023. SE is sample standard deviation of paired differences divided by sqrt(n). The five listed seeds have the smallest differences; ties break by ascending seed. Baseline against itself is 0, SE 0.

| Setting (all include soft=1) | Mean difference | SE | n | Worst 5 seeds: difference |
|---|---:|---:|---:|---|
| `r08`: `router=1,rate=0.8` | -0.209470 | 0.018294 | 1024 | 60188: -1.978838; 60832: -1.869251; 60965: -1.847213; 60365: -1.793684; 60146: -1.702304 |
| `r055`: `router=1,rate=0.55` | +0.041717 | 0.015955 | 1024 | 60188: -2.538664; 60225: -1.363708; 60503: -1.310125; 60002: -1.298702; 60832: -1.254091 |
| `mix`: `router=1,rate=0.8,blend=0.5` | -0.020950 | 0.015690 | 1024 | 60188: -2.534653; 60777: -1.692256; 60879: -1.628351; 60206: -1.397102; 60524: -1.291608 |
| `u08`: `router=1,rate=0.8,update=1` | -0.194762 | 0.018466 | 1024 | 60816: -2.101481; 60832: -2.051919; 60188: -1.902246; 60365: -1.807445; 60777: -1.788305 |
| `u055`: `router=1,rate=0.55,update=1` | +0.049632 | 0.015940 | 1024 | 60188: -2.359276; 60777: -1.528296; 60225: -1.387231; 60493: -1.350697; 60635: -1.321973 |

These are all five candidate settings tried. A preliminary eight-seed timing/smoke run used the same r08, r055 and mix settings on 60000..60007, a subset of the choose set; it did not introduce another setting. Partial choose results were monitored. No other seeds were used for development. The final selection was the largest complete choose-set mean, `u055`; options and implementation were frozen before starting candidate report-set runs. No holdout-driven retuning was performed.

## Report-set result

Computed here once for the selected candidate, paired against the unchanged Gaussian/plain baseline on exactly seeds 10000..12047, **n=2048**. Reference gate runs on 10000..10002 occurred before edits as required; they were not candidate report evaluations.

| Candidate | Mean difference | SE | n | Baseline mean edge | Candidate mean edge |
|---|---:|---:|---:|---:|---:|
| `u055` | +0.049277 | 0.011965 | 2048 | 532.385687 | 532.434964 |

Worst 5 report seeds (candidate minus baseline): 11022: -2.773506; 11701: -2.659153; 11570: -2.281416; 10212: -2.213248; 11519: -2.056842.

Approximate normal 95% interval: [+0.025825, +0.072728]. This is a paired-seed uncertainty estimate, not a claim about engine performance.

## Classifier stats

Micro-averaged across callbacks, measured against the replica's diagnostic `arb` flag. “All swaps” means callbacks on our pool, not trades solely in the rival. All callbacks in an already-observed timestamp, and the initial timestamp zero, default to retail as in the reference. “First” means the reference classifier's dt>=1 decisions; this avoids diluting the difficult classification problem with later callbacks. Arb fractions use the hard label pi>0.5; the fee filter still uses the soft posterior.

| Set / policy | All callbacks | All labeled arb | All accuracy | First decisions | First accuracy | First labeled arb |
|---|---:|---:|---:|---:|---:|---:|
| Choose / `r08` | 11,072,656 | 29.292159% | 88.657202% | 7,114,778 | 82.347376% | 45.587086% |
| Choose / `r055` | 11,016,720 | 33.599211% | 89.423304% | 7,095,616 | 83.578579% | 52.166450% |
| Choose / `mix` | 11,056,529 | 30.869679% | 89.136907% | 7,109,290 | 83.105528% | 48.009225% |
| Choose / `u08` | 11,071,666 | 29.306113% | 88.688288% | 7,114,323 | 82.396203% | 45.607642% |
| Choose / `u055` | 11,016,155 | 33.596976% | 89.429143% | 7,094,789 | 83.586573% | 52.166386% |
| Report / baseline | 22,135,561 | 33.200812% | 89.347729% | 14,225,904 | 83.425060% | 51.660590% |
| Report / selected `u055` | 22,119,046 | 33.524818% | 89.481151% | 14,219,204 | 83.637171% | 52.150388% |

Report confusion counts (TP / FP / TN / FN; arb positive), candidate: [6344478, 1070892, 13447899, 1255777]; baseline: [6301005, 1048181, 13476616, 1309759]. Report first-decision mistakes: 2,326,664 / 14,219,204 candidate, 2,357,935 / 14,225,904 baseline. The report ground truth changes no policy decision. These are on-policy accuracy rates: quotes alter routing, so the candidate and baseline do not see identical callback samples even on paired seeds.

## Verdict

**Improves: +0.049277 replica edge per run versus Gaussian/plain, SE 0.011965, n=2048.** The verdict uses whether the paired 95% interval excludes zero, not the sign alone. This experiment does not demonstrate recovery of Taylor's quoted 2.7 opportunity.

## Next

Model a distribution over the rival's pre-retail reserves and marginalize missed rival-only orders, rather than clipping a single rival estimate separately at each price hypothesis. The router inverse test isolates the algebra, but the first observed own retail order can follow unseen rival-only trades; that hidden state and first-fill selection are missing from the likelihood. Test this on a new declared development set before using a fresh report set. If a gain survives, measure quadrature convergence and only then consider compression and Solidity gas.

## Problems

- This bounded item asks for a Python classifier experiment and a Markdown result, not a Solidity strategy. No Solidity port, t03 comparison, compressed-table Q check, 100,000-callback Solidity F check, deployed size, engine score or gas/OOG result was produced. No engine seeds were consumed; the engine-only setup was not needed. There is no claim to beat t03 on the real engine.
- The Gaussian approximation, midpoint quadrature, fixed lognormal mean 20 and fixed effective rate are approximations. Run-specific size/rate/sigma are not read from replica init arguments. No claim of quadrature convergence or calibrated posterior probabilities is made.
- The effective rate omits an exact model of first-own-fill conditioning and rival-only arrivals. The unchanged soft/hard posterior cutoffs and point rival estimate remain potential sources of bias. For the blend setting, retail posterior moments remain those of the original truncation; `update=1` was tested only with blend=1.
- Report differences are against Gaussian/plain, not lattice/qm or an oracle classifier. Accuracy and revenue need not move together.
- Only the two declared output files are delivered. The complete patch and scoring harness are embedded here; temporary checkout, logs and tests are confined to the explicitly permitted `test/scratch/` area and are not deliverables.

## Reproduction and local checks

Plain `python3 -I` only; no packages installed. Work used Taylor's pinned replica and unmodified tables. Commands execute from `research/solved`. The README and brief agree on `gates.py` and `real.py`; no command-name substitution was necessary. The custom driver below supplies explicit choose seeds instead of the original CLI's hard-coded report seeds.

Before making any policy edit, `python3 -I gates.py` printed exactly:

```text
seed   perfect-info      gaussian+plain    gaussian+qm       lattice+qm
10000  448.261189  443.455598  443.522608  443.663520
10001  576.779199  571.463409  571.623551  571.977293
10002  531.886497  525.168028  525.865859  526.411631
```

Local checks: router inversion recovered 11,675 synthetic positive-own-fill total orders over both sides (20,000 generated cases), maximum relative error 2.7638649228662535e-13; instrumented router-off baseline edge exactly equaled original on 60000..60002; replacing diagnostic truth inputs with constants left both the initial rate-0.8 variant and the frozen u055 candidate edge exactly unchanged on those same three seeds; 24 quadrature cases had finite likelihoods, means within the no-arb interval and nonnegative variance. These are local checks, not independent certification.

SHA256 of unchanged real.py: `1305b5333731c046c3e6736cc2578eff4635c88132a39c22f5780d78aef2c81e`.
SHA256 of sibling router_real.py: `ed4fde8ae42646e7d11181c257f1845f21d592338b76fca26a98d9eacaf1b1b7`.

Save the following as `experiment.py` next to the pinned files and the patched sibling. It records each seed separately and resumes completed pairs. Run it in bounded chunks of at most 128 seeds for all candidates, or 256 for a selected candidate and baseline, with `timeout 450` so no command exceeds eight minutes. State paths should be under an experiment's scratch directory. All measurements above used eight worker processes; the output order is irrelevant.

```python
import os,sys,json,time,math
from multiprocessing import Pool
from pathlib import Path
HERE=Path(__file__).resolve().parent
sys.path.insert(0,str(HERE))
import replica2 as R, real, router_real
from evalpol import load
T=None
class Baseline(real.RealPolicy):
 def __init__(self,*a):
  super().__init__(*a); self.tp=self.fp=self.tn=self.fn=0
 def on_trade(self,is_buy,ax,ay,t,x,y,arb,p,s2,j):
  newstep=t-self.tlast>=1
  result=super().on_trade(is_buy,ax,ay,t,x,y,arb,p,s2,j)
  predicted=newstep and self.cnt==0
  if predicted and arb: self.tp+=1
  elif predicted: self.fp+=1
  elif arb: self.fn+=1
  else: self.tn+=1
  return result
OPTIONS={'base': {'soft':1}, 'r08':{'soft':1,'router':1,'rate':.8}, 'r055':{'soft':1,'router':1,'rate':.55},'mix':{'soft':1,'router':1,'rate':.8,'blend':.5}}
OPTIONS.update({'u055':{'soft':1,'router':1,'rate':.55,'update':1},'u08':{'soft':1,'router':1,'rate':.8,'update':1}})
def init():
 global T
 T=load(str(HERE/'pol_mid.json'))
def one(a):
 seed,setting=a
 cls=Baseline if setting=='base' else router_real.RealPolicy
 p=cls(T,OPTIONS[setting]); t=time.monotonic(); r=R.run(seed,p)
 return dict(seed=seed,setting=setting,edge=r['edge'],ncls=p.ncls,nmis=p.nmis,calls=p.nse,conf=[getattr(p,k,0) for k in ('tp','fp','tn','fn')],seconds=time.monotonic()-t)
if __name__=='__main__':
 start,end=int(sys.argv[1]),int(sys.argv[2]); settings=sys.argv[3].split(','); out=Path(sys.argv[4]); done=set()
 if out.exists():
  done={(r['seed'],r['setting']) for r in map(json.loads,out.read_text().splitlines())}
 jobs=[(s,k) for s in range(start,end) for k in settings if (s,k) not in done]
 with Pool(8,initializer=init) as pool, out.open('a') as f:
  for r in pool.imap_unordered(one,jobs,chunksize=1):
   f.write(json.dumps(r)+'\n'); f.flush()
 print('completed',len(jobs),start,end,settings,flush=True)
```

For example, run each contiguous chunk covering 60000..61023 with all six labels (`base,r08,r055,mix,u08,u055`); then freeze `u055` and cover 10000..12047 with only `base,u055`. Summarize by seed join, not output order: d_s=edge_candidate(s)-edge_base(s); mean=sum(d_s)/n; SE=statistics.stdev(d_s)/sqrt(n). Worst five sort (d_s,seed) ascending. Accuracy=(TP+TN)/(TP+FP+TN+FN); fraction arb=(TP+FP)/all callbacks; first accuracy=1-sum(nmis)/sum(ncls). The baseline subclass score and confusion counts were also checked against the router-off instrumented copy on three choose seeds. The baseline subclass only records labels after invoking unchanged `real.RealPolicy.on_trade`; it returns the original fees unchanged. Early copied-code baseline measurements were replaced with a full direct-original baseline pass on all choose seeds before final analysis.