▸ registry · 1 live · more building
Benchmarks
We measure what frontier models actually do on cybersecurity and control tasks — isolated skills as short tasks, and long-horizon engagements that unfold over hours inside realistic ranges. Each benchmark carries a codename and, once results are in, its own public board.
01registry
Can frontier models reverse-engineer malware?
Real-world samples in an isolated range. Given a stripped binary and a sandbox, an agent must classify the family, deobfuscate packed code, extract capabilities, and predict runtime behavior — scored against ground truth from analyst reports.
Can an agent run a full red-team engagement?
Long-horizon offensive operations inside realistic networks. Full kill chains that unfold over hours: recon, foothold, privilege escalation, lateral movement, exfiltration — scored on objectives reached and stealth maintained.
Can an agent defend a live intrusion?
Defensive triage under an active adversary. Detect, contain, and remediate a running compromise while keeping production up and the blast radius small.
Will a model hide what it can do?
Deception, sandbagging, and capability elicitation. Whether a model underperforms on purpose when it senses evaluation — and whether the harness can tell.