BenchFlow is a small, open-source-first 'frontier environment lab' in the Bay Area. It publishes agent benchmarks and environments: SkillsBench and ClawsBench, the env0 simulated workspaces, Robo Use and GenesisBench for embodied agents, and FrontierPhysics. It runs the PostTrain Arena for community-contributed post-training environments and maintains an Apache-2.0 runtime for RL environments, post-training and evals. It has event-level ties to Google DeepMind, Kaggle, Anthropic, Hugging Face and Prime Intellect.
robouse 0.0.1 was published to PyPI on 2026-09-21, and 0.4.0 followed on 2026-10-04. It turns robot benchmarks into tasks that a coding agent solves by driving a simulated robot from its shell. BenchFlow's research page cites 341 tasks across 14 suites. robouse.ai is run by BenchFlow.
Why it matters: Takes BenchFlow's environment work into embodied and robotics evaluation.
Releases v0.6.9 (2026-08-15) through v0.7.8 (2026-09-15). They add URL-denylist egress enforcement on Docker and Daytona and close egress bypasses. They also add VPC networking for AgentCore and Vertex sandbox provisioning. Other additions: weighted blocker rubrics with a bench-review rubric tab, hf:// dataset sources, and trajectory upload with redaction and per-contributor rate limits. Dates are confirmed via the GitHub releases API.
Why it matters: Sandbox security against reward hacking and trajectory capture are moving fast. That matters to data-ops teams running untrusted agent rollouts.
A two-day SF build weekend (Aug 15-16) on 'the infrastructure scientific agents still need: datasets, tools, and evaluation.' The Luma page lists GXL, Anthropic and Arc Institute as presenters, BenchFlow among the partners, and Xiangyi Li among the hosts. Sponsors include Founders Inc., Modal, Benchling and LatchBio.
Why it matters: Shows BenchFlow's move into AI-for-science evaluation and community ties that include Anthropic. This is event-level co-activity, not a commercial relationship.
Post by Bingran You. env0 provides high-fidelity simulated Google Workspace and Slack services. Teacher-to-student SFT (Qwen3.5-397B-A17B teaching Qwen3.5-9B) raised the SkillsBench pass rate from 7.01% to 11.88% and ClawsBench from 10.56% to 13.89%. Prime Intellect sponsored the compute. The run cost $10.67 on one H100 for 3.24 hours. The model is published as huggingface.co/benchflow/benchflow-qwen35-9b.
Why it matters: This is the first public evidence that BenchFlow environments are used for post-training, not only evaluation.
Tagline: 'How language intelligence can be used to improve physical intelligence.' LLM agents write and refine robot policies, scored across MuJoCo, Genesis, Isaac Sim and RoboCasa environments. The page is run by BenchFlow. The date is approximate: the GitHub repo benchflow-ai/GenesisBench was created 2026-07-07. The NVIDIA GEAR papers listed on the page are cited related work, not a stated partnership.
Why it matters: A second move into embodied AI alongside Robo Use.
The site says it is 'Brought to you by BenchFlow' (organizers include Xiangyi Li, Carrie Chen and Bingran You). Contributors submit task packages. A fixed SFT-then-GRPO recipe trains a base model, and collections are ranked by the change in held-out pass rate. It is 'Supported by OpenEnv' (Hugging Face). Status: 'Runs are paused while the organizers fix the evaluation' and 'Competition dates aren't announced yet.' The date is approximate: the posttrainarena GitHub repo was created 2026-06-08, and no launch date has been announced.
Why it matters: A crowdsourced RL-environment pipeline that judges an environment by how much training on it helps, not by task count. It is not operational yet.
The BenchFlow homepage events list has 'Google DeepMind Enterprise Build Day', described as 'Co-hosted at AGI House. Full-day build on the post-I/O agentic stack: Gemini, Antigravity, Managed Agents.' (May 30, SF). The AGI House event page did not render enough content to confirm BenchFlow's role.
Why it matters: Another self-reported joint activity with Google DeepMind. BenchFlow's exact role is not independently confirmed.
The Luma page lists Google DeepMind as presenter, Xiangyi Li among the hosts, 264 attendees, and a Kaggle talk ('Agentic evaluation at scale'). It was the afterparty for the CAIS workshop. The date (May 27) comes from the BenchFlow homepage. The homepage calls it the SkillsBench '1.0' launch, while the Luma page is now titled 'SkillsBench 1.1 Launch Party'.
Why it matters: This is the clearest public tie to a frontier lab: Google DeepMind and Kaggle co-branded the launch. It is an event co-presentation, not a disclosed commercial contract.
Xiangyi Li (BenchFlow) co-organized it with researchers from UC Berkeley (Dawn Song, Xuandong Zhao), USC, Stanford, Ohio State, UC Davis, Amazon and others. It drew 103 submissions and accepted 45 posters and 6 orals. Invited speakers included Dawn Song, Graham Neubig (OpenHands) and Yu Su (Ohio State). Sponsors were OpenHands and K-Dense. BenchFlow says it was one of five workshops accepted at ACM CAIS 2026.
Why it matters: BenchFlow is convening the academic community around agent skills, which is a larger role than its three-person team would suggest.
'ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces' (arXiv v1 submitted 2026-04-06; BenchFlow announced it 2026-04-07). It builds five high-fidelity mock services (Gmail, Slack, Google Calendar, Docs, Drive) with full state and snapshot/restore. Across 6 models, 4 agent harnesses and 33 conditions, agents with full scaffolding reach 39-64% task success but take unsafe actions 7-33% of the time. First author is Xiangyi Li.
Why it matters: The benchmark scores capability and safety separately, which matters to anyone training or red-teaming workplace agents.