Robotics B: five-event motion timeline (open environment)
Too easy: a strong agent hit 1.0 by writing its own OpenCV analysis. A 10-second clip can't be hard.
The ask
Same rollout, five motion events (motion_start, grasp, transport_apex, release, motion_end) in an OPEN environment: shell, internet, pip. The agent is told to bring its own tools.
Ground truth[logged]
Gripper channel plus end-effector height from forward kinematics on the 7 logged joint angles (Panda DH parameters). The FK was validated: the height curve descends to the logged grasp, peaks mid-transport, descends to the logged release.
Scorer
Deterministic comparison of event times against logged/FK-derived anchors.
The lesson
Agents self-tool when the environment is open — this one wrote its own OpenCV analysis (37 cv2 calls) rather than eyeballing frames. Real difficulty needs a long-horizon multi-step rollout; a 10-second pick-place cannot carry it.
Measured, not claimed
- Oracle
- 1.0
- Empty baseline
- 0.0
- Strong agent (Opus 4.8)
- 1.0
- Rollout
- 72 tool-call turns