The security paradox of local LLMs
Local LLMs prioritize privacy over security. Our research reveals a 95% backdoor injection success rate.
Jacek Migdal
Read more
Make AI agents
production-ready through
realistic simulations
Independent evaluation and training for the AI agent ecosystem. Real-world complexity through simulation environments where agents face multi-hour tasks.
Talk to FounderFor frontier labs
Large-scale RL datasets with tuned difficulty distributions. Cheat-proof reward functions. Teach skills scarce in public data (e.g. dependency hell, distributed system debugging).
For AI app developers
Measure quality and uncover blind spots. Pick optimal models, tune prompts in a fast-changing world. Benchmark against competitors. Win deals and deliver on performance promises.
For enterprise buyers
Independent verification of what actually works. Design processes based on real capabilities, not marketing hype. ROI-driven deployment decisions. Move from FOMO to measurable P&L impact.
Latest insights
Explore our research on AI agents, benchmarking, and evaluation
AI for coding is still playing Go, not StarCraft
AI excels at clean algorithms but fails at messy, real-world codebases. The solution lies not in Go-like intelligence, but StarCraft-like complexity.
Piotr Migdaล
Read more
Featured
CompileBench: Can AI Compile 22-year-old Code?
We tested 19 LLMs on their ability to handle real-world software engineering tasks like compiling old code and cross-compiling. See how Anthropic, OpenAI, and Google models stack up in our new benchmark โ CompileBench.
Piotr Grabowski
Read more
Looking for Quesma Gateway?
The Quesma database gateway IP has been acquired by Hydrolix to ensure continued support.
Read the announcement.