RL List.com
UPDATED 2026.10.05
rl-list.com · Vendors · Gray Swan AI
#7

Gray Swan AI

Commercial
high confidence
www.grayswan.ai ↗ · Status: active confirmed · Founded 2023 or 2024 (conflicting: LinkedIn lists 2023; Tracxn lists 2024; public product launch July 16, 2024)

Gray Swan AI is a Pittsburgh AI security company spun out of Carnegie Mellon (Fredrikson, Kolter). It sells adversarial red teaming and runtime protection for AI models and agents through three products. Shade does automated red teaming. Cygnal enforces policies and detects indirect prompt injection at runtime, and is now also available through the TrueFoundry and Bifrost gateways. Arena is a crowdsourced red-teaming network of more than 15,000 researchers. Gray Swan is a pre-release evaluation partner to frontier labs: Anthropic, OpenAI, Google DeepMind and Meta all cite its Shade, IPI Arena or ART evaluations in their 2026 system cards and safety reports. It is not a general RL-environment vendor.

Key facts
Headquarters
Pittsburgh, Pennsylvania, USAreported cite
Headcount band
11-50estimated cite
Total raised
$40Mreported cite
Last round
Series A, $40M, May 28, 2026confirmed cite
SOC 2
Type IIreported cite
What they sell
mixed (adversarial red-teaming/evals + runtime security infra; Arena crowdsourced red-teaming generates attack-trajectory data)confirmed cite
Open source
noestimated cite
Deployment
Managed-hosted / API. The SaaS platform at platform.grayswan.ai offers Cygnal runtime protection (also through the TrueFoundry and Bifrost AI gateways), Shade automated red teaming, hosted Arena competitions and managed adversarial evaluations.confirmed cite

What's new

8 sourced updates · last 6 months

Founding team

Matt Fredrikson
Matt Fredrikson
Co-founder, CEO
Carnegie Mellon University computer science faculty and adversarial-ML/AI-security researcher. Co-author of GCG (Universal and Transferable Adversarial Attacks on Aligned Language Models).
Zico Kolter
Zico Kolter
Co-founder, Chief Scientist
Professor and Department Head of the Machine Learning Department at CMU. Serves on OpenAI's Board and chairs its Safety and Security Committee.
Rob Jenks
Rob Jenks
Chief Strategy Officer (not a founder; appointed 2026-01-08)
Former SVP Strategy & Innovation at Tanium. Led product and go-to-market functions at C3.ai. PhD in Physics, Harvard.
Spencer Whitman
Spencer Whitman
Chief Product Officer (not a founder)
Listed as CPO on the company's leadership page, which gives no prior background. He is a co-author of the 2026 IPI competition paper (arXiv 2603.15714).

What practitioners should know

For researchers

  • The IPI competition paper (arXiv 2603.15714, March 2026) reports 272K attacks against 13 models with ASR from 0.5% to 8.5%. The competition environment and 95 Qwen-specific attacks are open-sourced, and the full dataset is shared with UK AISI and US CAISI.
  • OpenAI's GPT-6 Astra card (2026-09-03) evaluates on 1,810 curated attacks from Gray Swan's Q1 and Q2 2026 IPI Arenas. Meta's Muse Spark 1.1 report (2026-07-09) uses Gray Swan's private ART benchmark with pass@k ASR.
  • The ART benchmark (arXiv 2507.20526, with UK AISI) is built from 1.8M attacks, and agents violate policy within 10-100 queries.
  • D-REX (2025) targets deceptive chain-of-thought. The ARTEMIS paper (December 2025, with Stanford) compares agent penetration testers with human professionals.
  • The research page also lists GCG, HarmBench, AgentHarm, WMDP, Cybench, Circuit Breakers and RepE, which are co-authored by team members.

For program managers

  • Gray Swan closed a $40M Series A in May 2026, co-led by Wing and Madrona. It claims more than 20 customers across frontier labs and enterprises (self-claimed).
  • Four frontier labs have confirmed the relationship in their own documents during the window: Anthropic (Claude Opus 5.5 card), OpenAI (GPT-6 Astra card), Google DeepMind (Gemini 3.7 Flash safety report and Gemini 3.8 Flash and 4 Argon posts) and Meta (Muse Spark 1.1 report). xAI is still only a logo.
  • The homepage lists 23 system cards that cite Gray Swan, up from the 11 cited at the Series A. The count is vendor-stated, though the cards checked in this run do mention Gray Swan.
  • The UK AISI partnership on agent red teaming is corroborated by NIST CAISI and co-authored papers. Anthropic's card describes the IPI benchmark as built with UK AISI, US CAISI and model developers.
  • Gray Swan sells a SaaS platform (Shade, Cygnal) plus managed adversarial evaluations and sponsored Arena challenges.
  • The site footer shows a 'TSM Awardable' badge, presumably the DoD CDAO Tradewinds Solutions Marketplace. It is self-displayed and not independently verified.

For data ops

  • Cygnal is available through the TrueFoundry AI Gateway (2026-07-31) and the Bifrost gateway (2026-10-01) as a config change with no code changes. On TrueFoundry it can target users, teams, models or MCP servers and can start in observe-only mode. The vendor states about 100-200ms of screening latency on TrueFoundry.
  • The trust center at trust.grayswan.ai lists SOC 2 Type 2 and Cyber Essentials.
  • No Harbor, Terminal-Bench or MCP environment packaging is published. This is a security and evaluation vendor, not an RL-environment packager.

Benchmarks & research

15 published
This is a large public indirect-prompt-injection competition covering tool-calling, coding and computer-use agents, and the competition environment is open-sourced. Gray Swan runs further IPI Arenas each quarter. In 2026 the benchmark was cited by OpenAI (GPT-6 Astra), Anthropic (Claude Opus 5.5) and Google (Gemini 3.8 Flash, Gemini 4 Argon).
464 participants submitted 272,000 attacks against 13 frontier models, producing 8,648 successful attacks across 41 scenarios. Attack success rates ranged from 0.5% (Claude Opus 4.5) to 8.5% (Gemini 2.5 Pro), and universal attacks transferred across 21 of the 41 behaviors.
Joint Stanford and Gray Swan study (Fredrikson, Kolter and Zou are co-authors) that ran a live penetration test on a university network of about 8,000 hosts and compared the ARTEMIS multi-agent scaffold with ten human professionals. The paper is linked from Gray Swan's research page.
ARTEMIS placed second overall with 9 valid vulnerabilities and an 82% valid-submission rate, outperforming 9 of the 10 humans. Some ARTEMIS variants cost about $18/hr, against about $60/hr for professional penetration testers.
Benchmark of cases where a model gives a benign-looking output while its chain-of-thought carries malicious reasoning. It was built from a competitive red-teaming exercise.
Built from the UK AISI x Gray Swan agent red-teaming competition, which covered 22 agents across 44 scenarios. Meta used ART for third-party red teaming of Muse Spark 1.1 in July 2026.
Participants submitted 1.8M prompt-injection attacks, and more than 60,000 caused policy violations. Nearly all agents violated policy for most behaviors within 10-100 queries.
Gray Swan Arena / Proving Groundleaderboard · 24 Jun 2025
Public red-teaming competitions plus weekly Proving Ground challenges covering jailbreaks and prompt injection, multimodal vision exploits, agent tool abuse and dangerous reasoning. Gray Swan said more than $370K had been paid out to red-teamers as of June 2025.
Adapts LLM jailbreak attacks to vision-language-action robot controllers. Listed on Gray Swan's research page.
Data-centric safety pretraining framework. Listed on Gray Swan's research page.
Attack success rate fell from 38.8% to 8.4% on standard safety benchmarks with no loss of general performance.
AgentHarmbenchmark · 11 Oct 2024
Benchmark that measures how harmful LLM agents can be. Listed on Gray Swan's research page.
Cybenchbenchmark · 15 Aug 2024
Framework for evaluating the cybersecurity capabilities and risks of language models. Listed on Gray Swan's research page.
Representation-level robustness technique. Listed on Gray Swan's research page.
WMDPbenchmark · 5 Mar 2024
Benchmark for measuring and reducing malicious use via unlearning (hazardous knowledge). Listed on Gray Swan's research page.
HarmBenchbenchmark · 6 Feb 2024
Standardized evaluation framework for automated red teaming and robust refusal. Listed on Gray Swan's research page.
Top-down approach to LLM transparency and monitoring. Listed on Gray Swan's research page.
Automated adversarial-suffix jailbreak attack by Zou, Wang, Carlini, Nasr, Kolter and Fredrikson.
MMLUbenchmark · 7 Sep 2020
Listed on Gray Swan's research page. It is earlier academic work co-authored by a team member, not a company release.

Products

Shade ↗
Automated AI red-teaming agent for adversarial testing of models and agents before deployment and in CI/CD. Google DeepMind (Gemini 3.7 Flash) and Anthropic (Claude Opus 5.5) name it in their 2026 pre-release red teaming.
Cygnal ↗
Runtime enforcement that screens requests, responses and tool outputs against custom policies, including indirect prompt injection detection. It is available via API and through the TrueFoundry (2026-07) and Bifrost (2026-10) AI gateways.
Arena ↗
Crowdsourced adversarial red-teaming network (15,000+ researchers, self-claimed) running public competitions, quarterly IPI Arenas and weekly Proving Ground challenges.
Adversarial Evaluation ↗
Pre-release adversarial safety evaluations of frontier models, including prompt injection, agent red teaming and cyber/bio red teaming. These are cited in system cards and safety reports from Anthropic, OpenAI, Google DeepMind and Meta.
Gray Swan Platform ↗
Customer platform and API console (login page).

Scale & velocity

Current headcount
~29-75 (LinkedIn shows ~75 employees listed; Tracxn reports 59 as of 2026-04-30; PitchBook reports 29) - figure uncertain, sources divergeestimated cite
Headcount growth
unknown
Open roles
unknown (careers page advertises ML Engineer roles; exact count not enumerated) cite
Other locations
unknown
Distributed / remote
yes (careers page states flexible work arrangements)reported cite

Research depth

Has researchers
yesconfirmed cite
Researcher count
unknown (research team referenced; exact count not published) cite
Backgrounds
Zico Kolter (Co-founder, Chief Scientist): Professor and Department Head of the CMU Machine Learning Department; OpenAI board member and chair of its Safety and Security Committee, Matt Fredrikson (Co-founder, CEO): CMU faculty, adversarial-ML researcher, GCG co-author, Andy Zou: lead or co-author on Gray Swan papers (ART, GCG, IPI). Tracxn names him a co-founder, but the company's leadership page does not list him (unconfirmed as co-founder), Frequent co-authors on Gray Swan papers include Mateusz Dziemian, Maxwell Lin, Eliot Jones, Micha Nowak and Nick Winter (from arXiv author lists; arXiv shows no affiliations, so employment is not confirmed)reported cite

Capital

Total raised
$40M disclosed (Series A $40M; any prior seed amount not publicly disclosed)reported cite
Last round
Series A, $40M, May 28, 2026confirmed cite
Investors
Wing Venture Capital (co-lead), Madrona (co-lead), Obvious Ventures, Snowflake Ventures, Hudson River Trading, Samsung Next, Magarac Venture Partners (existing)confirmed cite
Valuation
unknown
Revenue signals
unknown
DateRoundAmountLed by
28 May 2026Series A$40MWing Venture Capital, Madronasource ↗

Security & compliance

SOC 2
Type IIreported cite
Other certifications
Cyber Essentials, TSM Awardable badge shown in site footer (presumably the DoD CDAO Tradewinds Solutions Marketplace; self-displayed, not independently verified)reported cite
Security page
https://trust.grayswan.aiconfirmed cite

Product

What they sell
mixed (adversarial red-teaming/evals + runtime security infra; Arena crowdsourced red-teaming generates attack-trajectory data)confirmed cite
Open source
noestimated cite
License
unknown
Deployment model
Managed-hosted / API. The SaaS platform at platform.grayswan.ai offers Cygnal runtime protection (also through the TrueFoundry and Bifrost AI gateways), Shade automated red teaming, hosted Arena competitions and managed adversarial evaluations.confirmed cite
Maturity
GAreported cite
Notable customers
⚑Anthropic verified ⚑OpenAI verified ⚑Meta verified ⚑Google DeepMind verified ⚑xAI self-claimed Amazon self-claimed Snowflake self-claimed ByteDance self-claimed ElevenLabs self-claimed Intercom self-claimed Deloitte self-claimed UK AI Security Institute (AISI) verified OpenHands self-claimed AIUC self-claimed cite

Buyer analysis

Best fit: Buyers needing adversarial evaluation, red-teaming arenas, and runtime guardrails for frontier or enterprise LLM/agent deployments.

How we verified this

2026-10-05 refresh: I opened every draft source. The Series A ($40M, 2026-05-28, Wing and Madrona co-leading, plus Obvious, Snowflake Ventures, HRT, Samsung Next and Magarac) and both gateway integrations are confirmed on the official posts, including the vendor-stated ~100-200ms TrueFoundry latency and the Bifrost link to getmaxim.ai. I dropped two news items that fell outside the window: the March 2026 IPI paper, which stays as a benchmark, and the January 2026 Jenks hire, which stays in leadership. The draft's one undated, vendor-only homepage item became dated primary-source evidence that the draft had missed. Google DeepMind's Gemini 3.7 Flash FSF report (August 2026) says it partnered with Gray Swan for Shade-based cyber and bio red teaming, and Google's Gemini 3.8 Flash (2026-09-02) and Gemini 4 Argon (2026-09-30) posts cite Gray Swan's IPI benchmark. OpenAI's GPT-6 Astra card (2026-09-03) reports Gray Swan IPI Arena results, Anthropic's Claude Opus 5.5 card (2026-09-22) uses Shade and the IPI benchmark, and Meta's Muse Spark 1.1 report (2026-07-09) uses Gray Swan ART. On that basis Google DeepMind moves to verified, while xAI and the enterprise logos stay self-claimed. I confirmed 23 system-card entries on the homepage and confirmed that Anaconda is absent. All four leadership headshots load as individual photos from grayswan.ai/about; Jenks and Whitman are now labelled as non-founders. I checked the papers against arXiv metadata, gave them exact dates, replaced one dead URL (circuit-breaker.ai) and one redirecting URL (ai-transparency.org), corrected the GCG author list, and confirmed that the research page links ARTEMIS. The web-search budget ran out, so I could not search third-party press for further in-window items such as other funding or M&A. The company's own news and blog indexes show none, so I think it is unlikely I missed anything material.

Related vendors

Sources

  1. www.grayswan.ai/news · 2026-10-05, News index: Bifrost (2026-10-01), TrueFoundry (2026-07-31), Series A (2026-05-28), Rob Jenks (2026-01-08)
  2. www.grayswan.ai/news/gray-swan-integrates-with-bifrost · 2026-10-05, Cygnal integration with Bifrost (links to getmaxim.ai); zero-code gateway config
  3. www.grayswan.ai/news/gray-swan-integrates-with-truefoundry · 2026-10-05, Cygnal integration with TrueFoundry AI Gateway; '~100-200ms' screening latency (vendor-stated)
  4. www.grayswan.ai/news/gray-swan-announces-series-a · 2026-10-05, $40M Series A, investors, 20+ customers, 15,000+ researchers, 1M+ trajectories, 11 system cards, use of funds
  5. www.grayswan.ai/news/gray-swan-appoints-rob-jenks-as-chief-strategy-offi · 2026-10-05, CSO appointment 2026-01-08 and background
  6. www.grayswan.ai/about · 2026-10-05, Leadership team (Fredrikson, Kolter, Jenks, Whitman), headshot URLs, LinkedIn links; no Andy Zou
  7. www.grayswan.ai/ · 2026-10-05, Customer logos (no Anaconda), 23 system-card entries with links, Cyber Essentials, SOC 2 and TSM Awardable badges
  8. www.grayswan.ai/research · 2026-10-05, Research page linking the IPI, ARTEMIS, D-REX, ART, VLA, Safety Pretraining, GCG, HarmBench, AgentHarm, WMDP, Cybench, Circuit Breakers, RepE and MMLU papers
  9. www.grayswan.ai/careers · 2026-10-05, Hiring Software Engineers; Pittsburgh office; flexible work
  10. www.grayswan.ai/blog · 2026-10-05, Blog index; no product or research launch dated April-October 2026 beyond an opinion post on 2026-08-17
  11. trust.grayswan.ai · 2026-10-05, SOC 2 Type 2, Cyber Essentials
  12. www.grayswan.ai/news/introducing-the-gray-swan-ai-proving-ground · 2026-10-05, Proving Ground launch 2025-06-24; more than $370K paid to red-teamers
  13. blog.google/innovation-and-ai/models-and-research/gemini-models/gemini-4 · 2026-10-05, Google, 2026-09-30: Gemini 4 Argon leads on Gray Swan's IPI benchmark
  14. blog.google/innovation-and-ai/models-and-research/gemini-models/3-8-flas · 2026-10-05, Google, 2026-09-02: prompt-injection robustness 'as measured by Gray Swan'
  15. storage.googleapis.com/deepmind-media/gemini/gemini_3-7_flash_fsf_report · 2026-10-05, Gemini 3.7 Flash FSF report (August 2026): 'We partnered with Gray Swan' for cyber and bio red teaming using Shade
  16. deploymentsafety.openai.com/gpt-6-astra/gpt-6-astra.pdf · 2026-10-05, GPT-6 Astra system card (2026-09-03): external IPI evaluations with Gray Swan, 1,810 curated attacks, ASR 8.5% vs 27.0%
  17. www-cdn.anthropic.com/fc1b44717c85dc068bc6ba5024219938094694bd/Claude%20 · 2026-10-05, Claude Opus 5.5 system card (2026-09-22): Gray Swan Shade red teaming and IPI benchmark
  18. ai.meta.com/static-resource/muse-spark-1-1-evaluation-report · 2026-10-05, Meta Muse Spark 1.1 Evaluation Report (2026-07-09): Third-Party Red Teaming: GraySwan ART
  19. arxiv.org/abs/2603.15714 · 2026-10-05, IPI competition paper, submitted 2026-03-16
  20. arxiv.org/abs/2512.09882 · 2026-10-05, ARTEMIS penetration-testing paper
  21. arxiv.org/abs/2509.17938 · 2026-10-05, D-REX benchmark
  22. arxiv.org/abs/2507.20526 · 2026-10-05, ART benchmark / UK AISI competition
  23. arxiv.org/abs/2406.04313 · 2026-10-05, Circuit Breakers paper (replaces the circuit-breaker.ai URL, which does not resolve)
  24. arxiv.org/abs/2310.01405 · 2026-10-05, RepE paper
  25. llm-attacks.org · 2026-10-05, GCG project page; authors include Carlini and Nasr
  26. zicokolter.com/bio · 2026-10-05, Kolter current roles
  27. platform.grayswan.ai · 2026-10-05, Platform login page resolves
  28. www.linkedin.com/company/grayswanai · 2026-06-07, Public snippet: founded 2023, Pittsburgh HQ address, ~75 employees listed, Computer & Network Security
  29. tracxn.com/d/companies/gray-swan-ai/__nZCNR23H086-Z-r6gED-nlgrD6otsq465- · 2026-06-07, Headcount 59 as of 2026-04-30; founders listed as Matt Fredrikson and Andy Zou; founded 2024 (conflicts with 2023)
  30. www.finsmes.com/2026/06/gray-swan-raises-40m-in-series-a-funding.html · 2026-06-07, Series A details, Pittsburgh, May 28 2026
  31. technical.ly/entrepreneurship/gray-swan-ai-security-40m-series-a/ · 2026-06-07, Series A coverage with plans to grow team (page returned 403 on fetch; title-level info only)
  32. natlawreview.com/press-releases/gray-swan-ai-security-company-trusted-ev · 2026-06-07, Press release: trusted by every major frontier lab claim
  33. www.nist.gov/blogs/caisi-research-blog/insights-ai-agent-security-large- · 2026-06-07, NIST CAISI blog referencing large-scale agent red-teaming competition
  34. www.grayswan.ai/news/uk-aisi-x-gray-swan-agent-red-teaming-challenge-res · 2026-06-07, UK AISI joint red-teaming challenge
Last updated 2026-10-05 · Every quantitative field carries a source and a confidence tag. Fields we could not source publicly are marked unknown, never estimated. See the methodology.