alakazam · Action Assay

Introducing Action Assay.

Action Assay is a certification gate for action-labeled robot video: an independent inverse dynamics model reads the actions out of the pixels, and a calibrated gate checks that the video did what its label says. Every episode gets a certificate or a rejection with reasons.

What that took: a 237,422-parameter model trained for 65 cents of GPU time. Held-out action recovery is R² 0.945 on the arms and 0.988 on the grippers, the gate is calibrated on 30 held-out real episodes and 33 faithful re-renders, and in the control battery, shuffled and wrong-arm actions fail hard.

READ & DOWNLOAD

Technical report ↗ Blog post ↗ API: OpenAPI spec ships with the code Code private during the pilot program, access on request

action-labeled episode: video + 14-D label

↓

independent IDM reads the actions out of the pixels

↓

calibrated gate: clip and fixed windows

↓

CERTIFIED or rejected, with reasons

One certification run: what goes in, what is checked, what comes out.

Read the technical report ↗

ACTION ASSAY · MEASURED

held-out R², arm / gripper0.945 / 0.988

render bandcorr ≥ 0.646 mae ≤ 0.755

real bandcorr ≥ 0.922 mae ≤ 0.273

temporal shufflecorr 0.16 rejected

wrong-arm swapcorr -0.52 rejected

port parity, episode 16Δ ≤ 0.0004 video round-trip residual

every number traces to a logged artifact; thresholds are p5/p95 of held-out calibration sets.

protocol · three steps

Train the auditor on real demonstrations. The instrument trains on 260 real episodes (44,913 frame pairs, 16 epochs) on one A10G; the full run, video decode to saved weights, takes 29 minutes. Generated video never enters training.

Calibrate two bands, real and faithful-render. The real band comes from 30 held-out real episodes, the render band from 33 faithful re-renders of real trajectories, because renders carry about a 0.24 correlation appearance penalty even when the motion follows the label. A faithful re-render of a real episode therefore fails the real band and passes the render band. Render bands are per-generator.

Run the control battery before any certification: six scored runs of five types. Two real held-out positives pass, a faithful render passes its band, and the negative controls fail hard: temporal shuffle drops to corr 0.16 and a wrong-arm swap to corr -0.52, both rejected. One same-task mismatch control lands near threshold; the per-frame series separates it, and the report states it as a limitation.

Why certification also gates fixed windows: clip means can hide a local collapse. All seven spliced-action chimeras pass the clip-level check against the render band, and in the four v2 renders the per-frame error after the splice runs 2.0 to 3.3 times the pre-splice level. The clip mean hid it and the per-frame series exposed it, so the gate scores fixed windows as well as the whole clip.

Scope: Action Assay ships as a Python package and an HTTP API, with one reference instrument (bimanual AgileX Piper, 14-D). An instrument is per-embodiment and per-camera-rig, and render bands are additionally per-generator; new embodiments and new generators need their own calibration.

Alakazam · 2026 · instrument artifacts logged per section · hugo@alakazam.gg