# qm-net: fixed-point MLP compression of Taylor's (gtmcknight) qm tables

**Result: the target is NOT met.** The chosen setting (int16 5-32-32-1 ReLU net, 3,076 bytes) has a report loss of **+0.589 se 0.014** (n=2048), above the < 0.2 target. The 12 KB size target is met. Loss kept falling with each training round (1.34 → 0.82 → 0.60 on the choose seeds). Training ran out of time before it plateaued, so more training, width 48 trained to convergence, or separate nets per tag are the obvious next steps (see Problems).

Source: github.com/gtmcknight/amm-sim @ 4812a6382311624e1506aca131ca35c378f930d9, research/solved. Everything below was run under plain `python3 -I` with numpy 1.26.4 for training and decoding only.

## Gate P (before any change)
```
$ python3 -I gates.py
seed   perfect-info      gaussian+plain    gaussian+qm       lattice+qm
10000  448.261189  443.455598  443.522608  443.663520
10001  576.779199  571.463409  571.623551  571.977293
10002  531.886497  525.168028  525.865859  526.411631
```
The gaussian+qm column matches the README (443.522608 / 571.623551 / 525.865859). My per-seed harness gives the same values for the full tables on seeds 10000..10002 (443.5226083, 571.6235507, 525.8658593).

## Inventory (`qm_k1e-5.json`)
- Scalars: `W0`=160, `NW`=321, `N2`=49, `C2`=24, `FB`=6, `H_bps`=5.0, `lam` 0.8, `sigma` 9.45, `size` 20.0, `g` 0.053682…, `kappa` 1e-5.
- `S` (belief sd levels, bps): [0, 4, 8, 12, 16, 20, 26, 34].
- `W2L` (rival-displacement knots, bps, 49): -160,-130,-100,-80,-60,-50,-40,-30,-25,-20,-17,-14,-12,-11,…(step 1)…,11,12,14,17,20,25,30,40,50,60,80,100,130,160.
- Tables, each level × row × column (row i → w = (i-160)·5 bps, -800..+800):
  | name | shape | A range (bps) | fee F=A-w > 0 fraction | rows with any F>0 (w, bps) | max F |
  |---|---|---|---|---|---|
  | A0c/B0c | 8×321×13 (cols R=-30..30 step 5) | -32.67..800 | 0.523 | -800..50 | 782.4 |
  | A1s/B1s | 8×321×49 (cols W2L) | -60..800 | 0.531 | -800..145 | 782.4 |
  | A1c/B1c | 8×321×49 | -53.12..800 | 0.526 | -800..60 | 775.4 |
  | A2s/B2s | 8×321×49 | -60..800 | 0.531 | -800..145 | 775.7 |
  | A2c/B2c | 8×321×49 | -55.56..800 | 0.526 | -800..60 | 772.8 |
  Total 1,073,424 values (536,712 per side). The min of A-w is exactly 0 in every table, so A ≥ w holds at every node.
- How real.py reads them (`RealPolicy.lookup`): it picks sd bracket k with S[k] ≤ sd < S[k+1] (clamped, linear weight tt in [0,1]). The tag is `0c` in phase 0; in phases 1/2 it is `1s/1c/2s/2c` (`s` if the rival was synced this step). It computes pw = w/5+160 clamped to [0,320]. For phase 0, R/5 is clipped to ±6 and offset by +6. For phases 1/2 it uses `_pos2` (binary search in W2L). It then does bilinear interpolation in (row, col) at levels k and k+1, linear in tt. Ask fee = 1-exp(-(a-w)·1e-4) if a>w, else 0. Bid uses B with b > -w.

## Symmetry
Point mirror B[k][i][j] == A[k][320-i][ncol-1-j] holds exactly in every cell of every table except **one**: `1c`, level 6 (sd 26), i=160, j=24 (w=0, R=0, the self-mirror centre), |diff| = 0.71 bps. So only the ask side is stored, and the decoder makes B by the mirror. That one centre cell is the only symmetry error, and it is included in the measured loss because the reference uses the full A and B.

## Method
- Target: the fee surface F = A - w (≥ 0, about half zeros), ask side only, all 5 tags × 8 sd levels.
- One net for all tags: 5 inputs (w/100, R/100, phase∈{0,1,2}, sd/10, synced∈{0,1}), then 32 ReLU, 32 ReLU, 1 linear. This is fixed-net's 4-input layout plus a `synced` flag, because the s/c tables differ and share a phase. The sd input is live.
- Loss: weighted squared error on F>0 nodes and one-sided (relu(pred)²) on F=0 nodes. Weights = node visit mass from the gaussian+qm policy on seeds 60000..60119 (the bilinear and sd-linear corner weights of each lookup, symmetrised by the mirror because bid lookups read mirrored ask nodes), normalised to mean 1, plus a floor (0.1 or 0.01), renormalised. Training uses numpy Adam with batch 4096 and a cosine learning-rate schedule over a wall-clock budget, warm-restarted.
- "Fixed positive-w path trick" (variant `path`): above a per-tag w cutoff (0c 50, s 145, c 60 bps), every F is 0. Those rows are not trained and are decoded as fee 0. The chosen setting does **not** use it (`none`): with equal training it lost on the choose seeds (0.654 vs 0.601).
- Integer forward pass (bit-exact in decoder and scoring):
  1. Inputs are rounded to Q12 (x·4096).
  2. For each layer: h = (h·W + b) >> shift, using arithmetic shift (floor). Weights are int16 = round(Wf·2^shift), with shift = floor(log2(32767/max|Wf|)). Biases are int64 = round(bf·2^(12+shift)). ReLU is applied on hidden layers.
  3. Output is the Q12 fee in bps.
  Shifts of the chosen net: 12, 13, 13. Quantisation changed the training loss from 4.217 (float) to 4.105 (weighted MSE, bps²).
- Decode: at every grid node A = w + max(net,0) and B = mirror(A), written as a full qm JSON. Scoring uses real.py's own code (`real.ev`, `RealPolicy` with {'soft':1,'qm':decoded.json}), unchanged, per seed. Loss per seed = full-table edge minus decoded-table edge, same gaussian+qm filter.

## Exact byte layout (3,076 bytes, little-endian)
| offset | bytes | content |
|---|---|---|
| 0 | 4 | magic `QMN1` |
| 4 | 3 | u8 n_in=5, u8 width=32, u8 flags=0 (0 = no path cut) |
| 7 | 3 | i8 shifts per layer (12,13,13) |
| 10 | 8 | u16 W0=160, u16 NW=321, u16 N2=49, u8 FB=6, u8 H_bps=5 |
| 18 | 8 | u8 S levels (0,4,8,12,16,20,26,34) |
| 26 | 98 | i16 W2L[49] |
| 124 | 320 | W1 int16 [5][32] row-major (in, out) |
| 444 | 256 | b1 int64 [32] |
| 700 | 2048 | W2 int16 [32][32] |
| 2748 | 256 | b2 int64 [32] |
| 3004 | 64 | W3 int16 [32][1] |
| 3068 | 8 | b3 int64 |

sha256 b509fbe9a9b2131087b7d2c03297f90d3bf06e36e859c3fa23927d81b51e9c25. Biases are stored as int64 because at least one bias at Q(12+shift) exceeds the int32 range. Re-scaling would allow a smaller layout (about 260 bytes saved), but that was not done.

## Tradeoff (choose seeds 60000..61023, n=1024, loss = full − decoded, mean ± se)
| setting | bytes | training | choose loss | se |
|---|---|---|---|---|
| w32, path, floor 0.1 | 3,076 (+cut consts) | 400 s | 2.529 | 0.032 |
| w48, path, floor 0.1 | 6,084 | 400 s (fewer epochs) | 2.967 | 0.036 |
| w32, none, floor 0.1 | 3,076 | 400 s | 1.339 | 0.024 |
| w32, none, floor 0.01 (a2) | 3,076 | 420+400 s | 0.820 | 0.022 |
| w32, path, floor 0.01 (c2) | 3,076 | 420+400 s | 0.844 | 0.022 |
| **w32, none, floor 0.01 (a3)** | **3,076** | 420+400+360 s | **0.601** | 0.020 |
| w32, path, floor 0.01 (c3) | 3,076 | 420+400+360 s | 0.654 | 0.020 |

Width 48 was only tried at the weak floor and short budget; at equal wall time it ran fewer epochs and lost. It was not retried at floor 0.01 because time ran out, so the comparison is not a fair test of width.

## Chosen setting and report
Frozen: a3 (w32, no path cut, floor 0.01). Scored once on seeds 10000..12047:
- **loss +0.5891, se 0.0138, n 2048** (the full-table per-seed edges on 10000..10002 equal the gates values; the README gives the full tables +3.77 vs HA)
- worst5 (seed, loss): (10955, 3.075), (10401, 2.988), (11494, 2.855), (11778, 2.847), (11519, 2.798)
- Target < 0.2: **not met**.

## Lookup cost (one ask/bid pair)
- As scored (decoded grid + real.py interpolation): 2 sd levels × 4 corners × 2 sides = 16 table reads. A contract evaluating the net at nodes needs 16 net evaluations of 1,216 multiplies each, 19,456 multiplies in total. That is too much; precompute is not possible on-chain.
- Direct net evaluation at the continuous point (not what was scored, which slightly changes the interpolation): 2 forward passes (ask at (w,R), bid at mirrored (-w,-R)). Each pass is 5·32+32·32+32 = 1,216 multiplies, 1,216 int16 weight reads plus 65 bias reads, 3 shifts, 64 ReLU compares, and **0 divisions**. In total 2,432 multiplies. The only work the two passes can share is the first-layer terms for the sd, phase and sync inputs.

## Standalone decoder (stdlib + numpy)
Writes a JSON that `real.py pol_mid.json qm:soft=1,qm=<file>` loads (keys S, W0, NW, N2, C2, FB, H_bps, W2L, A0c…B2c). It was verified to reproduce the scored tables exactly (array-equal on all 10 tables).
```python
"""Standalone decoder (stdlib + numpy) for the qm-net blob: writes a qm JSON that real.py's qm variant loads.
usage: python3 -I qmnet_decode.py <blob.b64> <out.json>
Net: 5-32-32-1 ReLU, int16 weights, int64 biases, Q12 integer activations, arithmetic right shift per layer.
Inputs: w/100, R/100, phase (0,1,2), sd/10, synced (1 for the s tables). Output: ask fee A - w in bps (Q12), clipped at 0.
Bid tables are the point mirror of the ask tables: B[k][i][j] = A[k][NW-1-i][ncol-1-j]."""
import sys,json,struct,base64,numpy as np
b=base64.b64decode(open(sys.argv[1]).read().strip()); assert b[:4]==b'QMN1'
nin,H,_,s1,s2,s3=struct.unpack_from('<BBBbbb',b,4); W0,NW,N2,FB,HB=struct.unpack_from('<HHHBB',b,10)
S=[float(x) for x in b[18:26]]; W2L=[float(x) for x in struct.unpack_from('<49h',b,26)]; o=26+98
L=[];dims=[nin,H,H,1]
for a,c in zip(dims[:-1],dims[1:]):
    W=np.frombuffer(b,'<i2',a*c,o).reshape(a,c).astype(np.int64); o+=2*a*c
    B=np.frombuffer(b,'<i8',c,o).astype(np.int64); o+=8*c; L.append((W,B))
assert o==len(b)
def fwd(X):
    h=np.round(X*4096).astype(np.int64)
    for l,((W,B),sh) in enumerate(zip(L,(s1,s2,s3))):
        h=(h@W+B)>>sh
        if l<2: h=np.maximum(h,0)
    return h[:,0]/4096.0
w=(np.arange(NW)-W0)*HB
Q={'S':S,'W0':W0,'NW':NW,'N2':N2,'C2':N2//2,'FB':FB,'H_bps':float(HB),'W2L':W2L}
for tag,ph,sy in (('0c',0,0),('1s',1,1),('1c',1,0),('2s',2,1),('2c',2,0)):
    Rv=np.arange(-FB,FB+1)*float(HB) if ph==0 else np.array(W2L)
    k,i,j=np.meshgrid(np.arange(8),np.arange(NW),np.arange(len(Rv)),indexing='ij')
    X=np.stack([w[i]/100,Rv[j]/100,np.full(i.shape,ph),np.array(S)[k]/10,np.full(i.shape,sy)],-1).reshape(-1,5)
    A=w[None,:,None]+np.maximum(fwd(X).reshape(i.shape),0)
    Q['A'+tag]=np.round(A,6).tolist(); Q['B'+tag]=np.round(A[:,::-1,::-1],6).tolist()
json.dump(Q,open(sys.argv[2],'w'))
```
Regenerate: save the base64 below to `qmnet.b64`, then run `python3 -I qmnet_decode.py qmnet.b64 qm_net.json`.

## Data (base64, 3,076 bytes decoded)
```
UU1OMQUgAAwNDaAAQQExAAYFAAQIDBAUGiJg/37/nP+w/8T/zv/Y/+L/5//s/+//8v/0//X/9v/3//j/+f/6//v//P/9//7///8A
AAEAAgADAAQABQAGAAcACAAJAAoACwAMAA4AEQAUABkAHgAoADIAPABQAGQAggCgAFL/zOuk89XggRbvC64B1AgZ/ir7dwrFAh3/
wP1vAa4ADQnz+IkDN/hKDmECsP9oC/f0bxYYGPDvS++H5tcVEgHrD2QlQ97dCJ3ZBSUF4H7eAfUIG37bzu276z7/GPlcCVDU/tmR
CzSzggA/Chf0OguFImj75wyH2EbnRvPMyrD6PgJ7CgEA7+29D7PzrRIu/iX4Nwpk/k/DEPU0FZsG+Qar/zkL6w3K/0zxBgK+/CMF
hA9//oMTEf8KAgz+nP4cAgb+kQsKBrfybPRy/pwIiQg8CLMc4gLpAqkNufqPDYz7fAOC/6AMkP2T9K8V8ApXA1H+pQGB/7ULDgkk
/ZL6U/7J5v8AaPtB/XETh+orCpn9uBbTAJwFudTn8PUGtgCpCAf2Kv00/X8Vg/y2Dzz12wgW/Cz84gMeBjoGIAD5COYEx/s3AQAA
AADbZKH//////yXVXf//////lrcO//////9We84AAAAAAJGZkAAAAAAAmFjGAAAAAAD3VawAAAAAAFHSuQAAAAAAgL3YAAAAAAA+
c5wAAAAAABKKPgEAAAAAkAu8AAAAAACABOkAAAAAABUA1gAAAAAAZ6rvAAAAAADaiQABAAAAADFvdv7/////fdeyAAAAAAAv674A
AAAAAEKL6QAAAAAAHx21AAAAAABBMg0BAAAAAFd2mQAAAAAAwdIEAQAAAACiSbEAAAAAAIJfugAAAAAAPJT0AAAAAAA+NrUAAAAA
ABRq8/7/////s8Z4AAAAAACkIRYBAAAAAGokiffI/PMYwDfbLZMpmz5YBcUBNR2GAKsp3SUBBlQrfRUN5c0e7TCr29glJdjW7bMo
m/QT0UEs8P7+7Kcy7QXy37r58wNK7x/jjdiu6+cM+wJY+z/sX/gs6MHdavzxz4zhzD1K4Vni3Tcu6Vo3Azsq1H4zbxvM0bfyAv+k
3sTw9+tOA0L35t/S8Evl7uKTHPsPoQes6jIF8dOB5bv5q+Iw4L4q+efz6VBAGOUZKAMoSehuOn0pW+7vMk36suQD9vUIqAUq+LgD
xv9R9Vn4OPz+++z1G/o/9FDwKPjC+YsId/2eNNn8wfp6NIH9XzAILhP33y0+Fhj1QcUo/N3xdvepBnoB+/9/IzcNohhxDsnfufY3
76gGfwNKGiESWAEdBjoZ0RzZCfQM3gaOGxcGcQA8EmwBtiC/BSQSHPCTE4EQKhXF/pQB1xqOJ/wbPit1Nc4AeQUnHITvFB2GKLD8
ryEdF2/jEydSIdH8zCJH4HnhbSyi4h8V7yfqF70HSyIKAZUU7vNaCTwWxhX3Cz8Oa+R3AE35kguu+TIOkBQD9GwSaPsP7L4KXQvN
/icBkQSDCQ4Xi/spFPkFQQIo+mYTCPeCCG0CSAxJEnkT8Q2KFzH/3g1wAZwFHvnTBqgDEgjRE38DAQSNEE8GTQbsD48D0vgVEPoK
wgoFFZ4g3PnLEYX8ZAYD+6sI+QEuEVYQcAfq/cz0lPawC0UAxQXCJNH5nwwNCqbvHxdkCKUF+heU+5IK5QgMFXzvhAe/A9XymwrL
9kf/mfg4/JgUkPsgBTMBaAjf9Z739g0c/JAE9QqwALwGiAsaBpkNVAzuBtD+7yKNGq4hOPkG+EQGYvYYAzEDO/7OFI3+kAK1G7kd
yxeMECULHvq1+tEVp/p7CE0YVPd8Cs8OrerD/6II6vYJE2b7TP1SHxH7dSdbHPcQqAGlEtv53wxKBWn17RhSDw0ngxs13Gb5OfHZ
GaEBsA2WI1kFCxVQEwn98RakFdkKjyQI/QYB7SYvE6zpwiMVQI39YhRIAGcEFfpM7p4M1wbDCDcJBwcSDqcAgv39/qwGHw8d+ZUD
DwVe+k/7TgTdD5YWxQLOAWsRrxE7AWgSaBr4B0MaMvywDHvwwPr1BHMLQwndCIf85wJdAPMFUvP/FOcV3PpzD93+8gbHCLgWyvpY
BzQYIAWCEXn6nQSODgMDSvSxChH9HgcC84EKog4ZACcIiA8pAnr0S/yKDbIAkw9+/OUMRg0gGpIO7woYAgkHTgy1BH//xxpI/ID2
Rwpa+6H9nwpK9uYUdfhvA1oadA7mC04QXiH6/x8BBQ8PAJsU3Ai9+doPOgrs8m0Thg/77Z8Kpw1iDGkHjNzPxt0bWvLNBlwF1v5t
Ik3uzfezI4kpXxpZHRAP8vyJ898ZevqqJu8eoPYbLsAWRdccId8hyuczJJTqHdzlGwgDOyYqIB4gdQc9LGvtLuu6+R7pZP0I7Ezt
guPvLt8INwJ+9yn+8u9u6c7/MfSD9Iw1a+z79VxNnuKSMS8xQ+7WJ6A4+/L9ByD+kOpp7LYRWfsq9nwL6QP99zgQrfVB/lcAWxMT
B0cKwv/L+j8R8hYWDMcD8xCbCKkFkRTyHZYLYggfD44Nmdm/7qkKW/6UK0b+wPwoJMgnhRyRHETlafdGB4ANRgW5IHsjFga9HasI
o/JAFQUhoelWGUjt2vvMET/0wyFXDZwotfpEEpf7VxOo/jIB4ifbHQoe3xXIIWYImPD8Hpb5gBs1HcIDzBvAFqREABscGbkf9CMe
77b0EBIiCCcQ2yLF+cYItjW5+xwUDwFW+coCcw40FVsMCfip92z9nRPZ//0LdBCk/BEIdwdF9vwMSwzCDh4AUQvgGX0MnAob8c4L
dvT9/kIGEwPwGTwDIvBgGXcWKQpnCW8PlwRl/CoD9v9qEowjWvN3JEgeKfv7HXsRXRNCDCcBxPWXBXwOyO0jFzwtvftvE5UAc/2w
/679mwqTAnb7rAipBLoKDQnyFVEA7ROeBrcCvBHtH64HlQCCFfn/eAraDjYI/xVjAoTyvguq0ZgBYAip+KsFEfsaAH8EmwsyEOUZ
DCYW5zH7KQdnAEIXDxAH7HEO4AvdKrQTGwIfLcX9ix6AJQUehwaS4+ARrdXmCt0GHvh2Hyv+WAaRCrwkORJdG9QXmPPy/bkVWwoO
GPkZdwJiLf4NlP2FExsXcwrnH0/4DvuxFIn17ugQHlkFcwT7EZb++wuGDKH9tATqBV8WXP4s+fX8ZewIB3AFSg3FAhsD7g4TG3n9
qwiGC3oFhQ0pEHsLn//I9TX1bQRJzL8AwxNl9sIULQVe/ggJugVZHBsXQvtc9jIJ+xun/boRPwayBG8HrRidFdkSOx//D2EaSQ/3
CfYSXh/0EQoWFxd3/ugopgCtCpn2SvRp/4cRnw9KAvP1IvzW+ZsAPQDHDugMtgSqBx0X5hDODyURvRFWEfkHcAdvDdAGeQ2zEMgR
qv9HEX4ANO/e+g0OOvD69PT/zvEfGhYDzPPY9XbzH/qV/mv7DwVW8vclkeuB51sx4vt7MVc7wPSiMfEbHwI504n50fyq9mQexwNE
7u0akyFdHsMOo+sU+yQG/Byv+IIc6hYg9ysNuBff8z0cexr06QQaXu8N3AMt1wZnRlgnl3GHA6EgqgG9Gwj3pPfWEscMDBVZFnMT
pACv+Fcau+shEtcSWALOGeAaGd3YG7YMlttWBjz71/ocEdnRHLg3Cr0MHPpeC5P5NMozAQAAAAAAfd7//////0GI3P//////us1L
AQAAAADb4Z8BAAAAAA13dwEAAAAAQLEfAQAAAAAZz1UBAAAAABdK7///////9oXk//////8wogIBAAAAAHwl2f//////81AtAQAA
AABq2XQBAAAAAPX03///////I3cLAQAAAABmPQ8BAAAAADtUIP//////HqQhAQAAAACEwRgBAAAAAEoxIf//////rzqLAQAAAABb
a3z+//////VYRv7/////WXpZAQAAAAC7d6f//////xYQr/7/////TFeAAQAAAACRDH8AAAAAAGcTxf//////Af2GAQAAAABmtO7/
/////2nm5v+BAq7rI+nK6kXdP7Jb/0cDJODxAnzoxO6nBqjhROVHMmvj1uUYLAHmSTIvKxfw5SP0SMjp7FFKBSDsfQEa8jyBAQAA
AA==
```

## Problems
- **Target missed**: report loss 0.589 vs < 0.2. The fit is still improving with training time: weighted RMSE went 3.07 → 2.37 → 2.05 bps over the three rounds, and choose loss went 1.34 → 0.82 → 0.60. The limit was the 8-minute-per-command cap and a ~2 h budget for numpy training on 0.5M nodes, not a demonstrated capacity limit.
- Next steps that were not run: train to convergence (more warm restarts, or importance-sample nodes by visit mass instead of weighting); width 48 at floor 0.01; one net per s/c group (two w32 nets = 6.2 KB, still under 12 KB); fine-tune the loss directly on interpolated lookup points.
- The `synced` input makes the net 5-input, not fixed-net's 4-input. fixed-net's plain-table loss (-0.040) was not reproduced here.
- Two scoring runs were lost to the time cap or to CPU contention and were rerun. No seed set was used for both choosing and reporting. Visit weights came from seeds 60000..60119, which are part of the choose set.
- The single non-mirrored cell (1c, sd 26, centre) is unavoidable error under ask-only storage (0.71 bps at one node).