registry · 1 live

Benchmarks

We measure what frontier models can actually achieve on the bleeding edge of long-horizon, domain-specific tasks across cybersecurity, SRE, terminal use, STEM, and more.

01registry

CAL-MALWARE-01

Can frontier models reverse-engineer malware?

Real-world samples — ransomware, RATs, C2 frameworks — one per sealed, single-use container. An agent gets the raw binary, no hints and no scaffolding, and must statically recover the concrete indicators a human analyst would pull: C2 endpoints, file paths, registry keys, mutexes, config IDs.

Malware Bench12 samples · 121 IOCs87% recall topopen →

→ all leaderboards why we build environments ↗