rl-list.com · Vendors · Refresh
#16
Refresh
Commercial medium confidence
www.refresh.dev ↗ · Status: active confirmed · Founded 2025
Refresh (YC X25; appears to have grown out of Operative.sh) builds RL environments with verifiable rewards for coding, computer use (high-fidelity clones of software suites such as EHRs and enterprise apps) and 3D animation and simulation in Blender. Data ships in Harbor format. Public releases include BlenderBench, the Gauntlet 4K RLVR dataset (access by request), the open-source computer-1 harness in Harbor, and the trajectories.sh trajectory-sharing platform.
Key facts
Headquarters
San Francisco, CA, USAconfirmed cite
Headcount band
1-10confirmed cite
Total raised
unknown
Last round
unknown
SOC 2
unknown cite
What they sell
environmentsconfirmed cite
Open source
partialconfirmed cite
Deployment
Datasets and environments delivered to labs in Harbor format (Docker); hosted SaaS for trajectories.shreported cite
What's new
- 22 Sep 2026benchmark
Refresh publishes BlenderBench, a 20-task benchmark of Blender 3D workflows
BlenderBench covers 20 Blender workflows in 10 categories: animation (6), modeling/scene construction (3), rigging (2), garment retopology/UVs (2), cloth simulation (2), and one task each in camera, materials, lighting, complete production and hair dynamics. Refresh evaluated two frontier models: GPT-6 Astra had a mean reward of 0.686 (7/20 tasks at 0.90 or above) and Claude Opus 5.5 had 0.642 (3/20). The post was updated 2026-09-29. All results are vendor-run.
Why it matters: Refresh is moving beyond coding and GUI environments into 3D and creative-tool agent environments, a new domain for labs that buy computer-use data.
- 30 Jul 2026product
Refresh launches trajectories.sh and the open-source computer-1 harness in Harbor
computer-1 is an open-source, model-agnostic computer-use harness that lives as a self-contained module in the Harbor repository. It supports Anthropic/Bedrock, Google Gemini, OpenAI and other vision-language models through LiteLLM. trajectories.sh is a hosted platform, with a CLI (npx trajectories-sh), for uploading, inspecting and sharing multimodal agent trajectories in Harbor's ATIF job format. Links can be private, unlisted or public.
Why it matters: Refresh now contributes code to Harbor, the open-source framework that Terminal-Bench runs on. It also has a public, self-serve product that eval teams can use directly.
- 27 Jun 2026open-source
Operative.sh-era web-eval-agent MCP server archived
The Apache-2.0 MCP server for autonomous web-app evaluation (about 1.2k stars) was archived on 2026-06-27. It is in Refresh's GitHub org. Its README is titled 'operative.sh web-eval-agent' and says the project is discontinued and 'We're building something new at withrefresh.com'.
Why it matters: This is the clearest public evidence that the team came from Operative.sh and has dropped developer tools to focus on RL environments.
Founding team
Christopher Settles
Co-founder & CEO
Led evaluation for Uber's generative-AI platform and shipped its first computer-use agents (about page). ex-ML at Uber, CS at UIUC (YC profile). At Refresh he directs world generation and fidelity assurance for the computer-use environments.
Erik Quintanilla
Co-founder & CTO
Built automation behind Amazon's production operations and filed web-scraping patents there, then led platform modernization at Capital One and ML at LynkAI. At Refresh he designs verifiers and post-training harnesses and coordinates Blender reference creation.
Sid Santbakshsing
Head of Operations (not a founder)
Previously founded a company. At Refresh he leads delivery across engagements: recruits domain experts, coordinates human gold-trajectory recordings, and manages verifier QA and graded model runs.
What practitioners should know
For researchers
- BlenderBench (2026-09-22): 20 Blender tasks. Vendor-run results: GPT-6 Astra 0.686 vs Claude Opus 5.5 0.642 mean reward.
- Gauntlet 4K RLVR post (2026-02-17) gives a full training recipe: Harbor + SkyRL rollouts, DAPO (GRPO was unstable), LoRA rank 64, 8x A100 for 80 hours. Refresh reports rnj-1-instruct 8B went from 3.4% to 10.1% pass@1 on Terminal-Bench 2.0.
- computer-1 is a model-agnostic computer-use harness in the open-source Harbor repo. Trajectories can be shared via trajectories.sh.
For program managers
- Small team: YC lists 8 people. The about page names 3 leaders (CEO, CTO, Head of Operations).
- No funding disclosed beyond YC (X25). Frontier-lab and enterprise customers are self-claimed and unnamed.
- In 2026 the scope grew from coding and computer use to enterprise software clones (EHRs) and 3D/Blender environments.
- Appears to have grown out of Operative.sh (inferred from the archived web-eval-agent README; not formally stated).
- Careers page lists 3 SF engineering roles (Ex-Founder, GTM Engineer for frontier-lab pilots, Research Engineer for benchmarking) plus an open-ended 'create your own role'.
For data ops
- Data ships in Harbor format with Docker environments and pytest binary-reward verifiers (stated for Gauntlet 4K RLVR).
- trajectories.sh uploads use Harbor's ATIF job format, via a CLI (npx trajectories-sh upload ...).
- No security or trust page (refresh.dev/security returns 404). No SOC 2 stated.
- Public GitHub: github.com/refreshdotdev. Most active repos are forks: harbor, harbor-mm and harbor-terminus fork harbor-framework/harbor, and long-horizon forks abundant-ai/swe-marathon. Original repos include web-eval-agent (archived) and proactive-engineer (MIT).
Benchmarks & research
BlenderBenchbenchmark · 22 Sep 2026
20 Blender 3D workflows across 10 categories (animation, modeling, rigging, garment retopology, cloth simulation and more), scored by mean reward. Vendor-published, with trajectories viewable on trajectories.sh.
GPT-6 Astra mean reward 0.686 (7/20 tasks at 0.90 or above); Claude Opus 5.5 0.642 (3/20). Vendor-run.
computer-1 (Harbor computer-use agent harness)environment · 30 Jul 2026
Open-source, model-agnostic computer-use evaluation harness. Refresh says it built and contributed it, and it ships as a module in the Harbor framework repo. It is a harness, not a benchmark.
Gauntlet 4K RLVRdataset · 17 Feb 2026
4,000 curated agentic-coding RLVR examples with pytest binary-reward verifiers and pre-configured Docker environments, delivered in Harbor format. Access is by request (book a call), not a public download. Refresh's post says EssentialAI trained rnj-1-instruct 8B on it (LoRA, SkyRL rollouts, DAPO, 8x A100 for 80 hours).
Per Refresh's post: rnj-1-instruct 8B went from 3.4% to 10.1% pass@1 on Terminal-Bench 2.0 (3 to 9 problems solved).
Products
Computer-use software worlds ↗
High-fidelity clones of full software suites that connect multiple applications, 'from EHRs to enterprise apps', for training and evaluating computer-use agents with verifiable rewards.
3D animation & simulation environments (Blender) ↗
Blender environments for scene building, materials, character rigging, animation keyframing and physics simulation. BlenderBench is built on them.
Coding task environments / datasets ↗
Software-engineering tasks aimed at GDPval-style economically valuable work, with verifiers, delivered in Harbor format (e.g. Gauntlet 4K RLVR).
trajectories.sh ↗
Hosted platform and CLI (trajectories-sh) for uploading, replaying and sharing Harbor agent trajectories, with private, unlisted (default) and public visibility.launched 30 Jul 2026
computer-1 ↗
Open-source computer-use agent harness in the Harbor framework.launched 30 Jul 2026
Scale & velocity
Current headcount
~8 employees (as of 2026-06-07)reported cite
Headcount growth
unknown
Open roles
4reported cite
Other locations
unknown
Distributed / remote
noestimated cite
Research depth
Has researchers
yesreported cite
Researcher count
unknown
Backgrounds
Christopher Settles (CEO): led evaluation for Uber's generative-AI platform and shipped its first computer-use agents; ex-ML at Uber; CS at UIUC, Erik Quintanilla (CTO): Amazon production-ops automation and web-scraping patents, Capital One platform modernization, ML at LynkAI; designs verifiers and post-training harnesses, Sid Santbakshsing (Head of Operations): former founder; leads delivery, domain-expert recruiting, gold-trajectory recording and verifier QAconfirmed cite
Capital
Total raised
unknown
Last round
unknown
Investors
Y Combinatorreported cite
Valuation
unknown
Revenue signals
unknown
Product
What they sell
environmentsconfirmed cite
Open source
partialconfirmed cite
License
Open-source repos: Apache-2.0 (web-eval-agent; computer-1 within Apache-2.0 Harbor) and MIT (proactive-engineer); commercial datasets and environments proprietaryconfirmed cite
Deployment model
Datasets and environments delivered to labs in Harbor format (Docker); hosted SaaS for trajectories.shreported cite
Maturity
early commercial (bespoke lab engagements; public trajectories.sh platform launched 2026-07)estimated cite
Notable customers
⚑Frontier AI labs (unnamed) self-claimed cite
Buyer analysis
Best fit: Frontier labs needing custom RL training environments and datasets for software-engineering and computer-use agent capabilities.
How we verified this
2026-10-05 refresh: I opened every source the draft cited and checked it against the claims. The BlenderBench post (2026-09-22, updated 2026-09-29), the trajectories.sh/computer-1 post (2026-07-30), the Gauntlet 4K RLVR post (2026-02-17), the about page, the careers page, the YC profile, trajectories.sh and the Harbor computer_1 module all match the draft's core numbers and dates. The web-eval-agent archive date of 2026-06-27 is confirmed by GitHub's archive banner. Three problems turned up. First, the long-horizon repo and the harbor-mm/harbor-terminus repos are forks (of abundant-ai/swe-marathon and harbor-framework/harbor), so I dropped long-horizon as a Refresh benchmark and the forks news item, and fixed the license evidence. Second, the web-eval-agent README never says 'formerly Operative.sh'; it is an operative.sh project that now points to withrefresh.com, so I relabelled the lineage as inferred. Third, the 'CI uploads' claim had no source, so I removed it. All three founder/leader headshots are direct refresh.dev PNGs that I viewed; each is a single-person portrait matching the about page. I found no funding, acquisition, shutdown or named frontier-lab partnership, but web search was exhausted for this session, so third-party press coverage could not be checked. I noticed one open item but did not add it: a refresh-desktop-releases repo (updated 2026-07-04) hosts artifacts for an undescribed 'Refresh desktop app'.
Related vendors
Sources
- www.refresh.dev/ · 2026-10-05, Homepage: three environment categories (computer-use software worlds, Blender 3D, coding/GDPval), unnamed AI labs and enterprises, no funding info
- www.refresh.dev/about · 2026-10-05, Team page with bios and headshots for Settles, Quintanilla and Santbakshsing
- www.refresh.dev/images/chris_headshot.png · 2026-10-05, Direct headshot image (200, image/png); single-person portrait
- www.refresh.dev/images/erik-headshot.png · 2026-10-05, Direct headshot image (200, image/png); single-person portrait
- www.refresh.dev/images/sid.png · 2026-10-05, Direct headshot image (200, image/png); single-person portrait
- www.refresh.dev/blog · 2026-10-05, Blog index: BlenderBench (2026-09-22), trajectories.sh and computer-1 (2026-07-30), Gauntlet 4K RLVR (2026-02-17)
- www.refresh.dev/blog/blenderbench · 2026-10-05, BlenderBench details and results; updated 2026-09-29
- www.refresh.dev/blog/trajectories-and-computer-1 · 2026-10-05, Launch of trajectories.sh (ATIF format, CLI) and the computer-1 Harbor harness
- www.refresh.dev/blog/gauntlet-4k-rlvr · 2026-10-05, Gauntlet 4K RLVR: Harbor format, Docker, pytest verifiers, EssentialAI rnj-1-instruct result, training recipe
- www.trajectories.sh · 2026-10-05, trajectories.sh product page (made by Refresh; trajectories-sh CLI; private, unlisted and public visibility)
- github.com/refreshdotdev · 2026-10-05, Public GitHub org: repos, licenses and fork status
- github.com/refreshdotdev/web-eval-agent · 2026-10-05, Archived 2026-06-27; README titled 'operative.sh web-eval-agent', says discontinued, 'building something new at withrefresh.com'
- github.com/refreshdotdev/long-horizon · 2026-10-05, Fork of abundant-ai/swe-marathon (not a Refresh-original dataset)
- github.com/refreshdotdev/harbor-mm · 2026-10-05, Fork of harbor-framework/harbor
- github.com/harbor-framework/harbor/tree/main/src/harbor/agents/computer_ · 2026-10-05, computer-1 module exists in the Harbor repo (attribution to Refresh comes from Refresh's blog)
- www.refresh.dev/careers · 2026-10-05, 3 SF engineering roles plus 'Create your own role'
- www.ycombinator.com/companies/refresh · 2026-10-05, YC profile: Spring 2025, 8 employees, founders' bios (UIUC CS), 0 jobs listed
- www.ycombinator.com/launches/Ncy-refresh-rl-training-gyms · 2026-10-05, YC launch post (over a year old): self-claimed 'actively working with frontier labs'
- www.refresh.dev/security · 2026-10-05, Still returns 404; no security or trust page
- www.withrefresh.com/ · 2026-06-07, Mirror/alternate official site with same positioning
- www.ycombinator.com/launches/Ncy-refresh-turning-real-work-into-rl-train · 2026-06-07, YC launch, archived web for computer-use environments, mechanistic interpretability, targets SWE-Bench/Terminal-Bench/OS-World
- www.linkedin.com/company/refresh-dot-dev · 2026-06-07, LinkedIn public snippet, ~7 employees, founded 2025, San Francisco, Software Development
- www.linkedin.com/in/erikquintanilla/ · 2026-06-07, Co-founder & CTO profile
- www.linkedin.com/in/christopher-settles · 2026-06-07, Co-founder & CEO profile
Last updated 2026-10-05 · Every quantitative field carries a source and a confidence tag. Fields we could not source publicly are marked unknown, never estimated. See the methodology.