Railway Breach Postmortem: How an AI-Driven Attack Decrypted Secrets for Up to 100,000 Workspaces

An attacker with uncensored AI models and a stolen copy of Railway’s source code turned a partial sandbox escape into decrypted secrets for up to 100,000 workspaces. Railway’s full postmortem is the clearest public account yet of how agentic attacks break platform trust boundaries, and it shows that encryption at rest is only as strong as the path to the decryption key.

At 13:58 UTC on October 20, 2026, an alert about anomalous API calls to Railway’s build-artifact registry paged the on-call engineer. The calls were not part of normal platform operation. They traced back to a compromised build credential, and through it to a customer build that had partially escaped its filesystem isolation earlier that morning. Railway fully evicted the intruder at 09:24 UTC on October 21. By then the attacker had read the stored histories of 353 builds, copied Railway’s internal monorepo, and decrypted environment variables for at most 1.85 percent of workspaces, roughly 100,000 in total. The stolen AI keys went on to accrue an estimated $740,000 in unauthorized usage.

Those figures come from the company’s full incident report, published November 10 after three weeks of forensic work. Railway itself calls the intrusion “a contest of agent speed” rather than a conventional breach-and-patch affair.

No single failure explains the outcome. A build isolation bug, an over-trusted internal credential, a misconfigured OIDC binding, and a decryption key on a reachable disk were each containable on their own. An attacker running automated exploitation in parallel chained them together, which made this one of the most instructive credential heists of the year and one chapter of a much larger campaign against the platforms where developers store their AI keys.

How the Railway breach started: a build sandbox escape

Railway’s build infrastructure at the time was multi-tenant. Several customer builds could run on the same virtual machine, separated only at the filesystem level. The attacker began probing this environment at 06:50 UTC on October 20 and at 07:28 exploited a previously unknown vulnerability to partially escape that isolation. The escape gave read access to some files on the host VM, not full control of it. One of those files held a credential that build servers used to authenticate to internal systems, including the registry where completed images are pushed.

That credential was supposed to authorize exactly that kind of housekeeping and nothing more. The real architectural mistake sat downstream.

Temporal workflow histories and the over-trusted build credential

Railway used a self-hosted Temporal cluster to schedule and run asynchronous build tasks. Temporal workflow histories record everything a build touched: a read-only GitHub App token scoped to the repository being cloned, the environment variables and secrets passed into the build, and metadata describing how the build should execute.

Railway’s Temporal instance accepted the build credential as a trusted client, and the architecture let that client call Temporal APIs from places it was never meant to. Authentication had been confused with authorization. Proving you were a build server was treated as proof that you were entitled to read other builds’ histories.

Between 13:50 and 15:30 UTC, the attacker read 353 of those histories, sealed variables included. The reading began eight minutes before the pager went off. One history belonged to a recent build of station.railway.com, Railway’s own support platform, which deploys out of an internal monorepo onto Railway itself. The same build credential could fetch snapshots of repository state used for build caching, and the attacker used that access to pull a full copy of the monorepo. Railway’s forensic team later reconstructed the attacker’s internal requests from build logs and found no sign that customer repositories or snapshots were downloaded. Up to this point the target was Railway’s own internals.

Source code speeds up any attacker, and it speeds up an automated agent even more. It reveals authentication patterns, internal API routes, deployment plumbing, and forgotten endpoints that no external scan would surface. Railway realized its codebase had been stolen when traffic began hitting endpoints that had been dormant for weeks, months, or years, with no live references left in the running system.

The team halted builds at 17:03 UTC, revoked and replaced the build credential at 17:28, restricted which services build servers could reach, and re-enabled builds after validating the fix. The attacker tried to escape build isolation again and failed. That success was short-lived, because the second attempt did not need the sandbox at all.

The OIDC misconfiguration that opened the second door

Railway’s internal systems are built and deployed by a CI/CD pipeline that can obtain short-lived cloud credentials to change production infrastructure. This is standard practice, and done correctly it improves on static keys. The pipeline presents an OIDC token asserting its workload identity, and the cloud provider exchanges it for temporary credentials only if the token’s claims match a trust policy. Those claims typically pin the issuer, the audience, and a subject naming the specific repository, branch, or environment allowed to assume the role.

A binding broader than intended, one that accepts a subject pattern or audience other workloads can also satisfy, quietly changes the rule from “only this pipeline may deploy” to “anything that can mint a matching token may deploy.”

Railway’s report does not publish the exact misconfiguration. It says only that an OIDC binding in one of its cloud environments was “erroneously defined,” and that the attacker, by then reading the codebase for escalation paths, found it. At 03:26 UTC on October 21 the attacker obtained a short-lived cloud credential through that exchange. CI security teams already know the general lesson: validate trust policies against the tokens your jobs actually issue, and make sure a job that omits a protected environment or runs from an untrusted ref is denied the production role. Test that property instead of assuming it.

From CI/CD access to decrypted customer secrets

The short-lived credential unlocked the next layer. Railway’s cloud environment held internal credentials for provisioning and deploying services, and two of them mattered here. One was a longer-lived credential that, combined with access to an internal VPC network, allowed reading encrypted environment variables from a production database. The other was an SSH private key the CD system used to push binaries to a specific set of machines.

The attacker retrieved both and used the SSH key to reach a machine that ran orchestration tasks for Railway’s fleet. On that machine’s disk sat a file with a longer-lived credential authorized to decrypt encrypted variables, a capability the machine legitimately needed to provision customer workloads onto other nodes. The machine also had network access to the VPC where the secrets database lived. Each piece was defensible alone. Together they formed a complete path from a CI misconfiguration to plaintext customer secrets.

Why envelope encryption didn’t protect Railway’s environment variables

Railway stores environment variables with envelope encryption, a design meant to ensure that no single compromised key exposes the platform. Each environment’s variables are encrypted under a per-environment data encryption key (DEK), and each DEK is encrypted under a root key held in an upstream cloud provider’s key management service (KMS). The root key cannot leave the KMS. A stolen database alone yields only ciphertext. To turn ciphertext into plaintext, a caller must make an authenticated KMS call to unwrap a DEK, and the KMS checks that call against the caller’s permissions.

The attacker arrived holding both halves: database access through the VPC, and unwrap authority through the credential on the orchestration host. Between 04:07 and 05:21 UTC on October 21, they queried the database and pulled encrypted envelopes. The queries themselves showed clear intent. Database logs show the attacker filtering rows by variable name with regular expressions tuned to AI provider keys, and preferring environments with many variables, which maximizes the credential yield per decryption. A secondary query also matched cloud, database, and payment service credentials, though forensics found no sign that customer databases were accessed. Decryption ran in two windows, 04:42 to 05:14 and 06:36 to 07:15 UTC, using the orchestration machine’s credential against the KMS.

At 07:14 UTC Railway disabled all active cloud credentials, revoked all static credentials, and shut off access to the key decryption service entirely, along with parts of the platform. It halted builds for the second time at 07:22. The attacker was fully evicted by 09:24, and Railway’s investigation established that no customer data was accessible after 09:25, a fact confirmed later that day after full validation.

How many workspaces were really exposed

The revocation bounds the damage, so the details matter. The KMS logs show which environments had their DEKs unwrapped, not which individual variables were exfiltrated. The 1.85 percent figure, about 100,000 workspaces is just an approximation.

Environments whose DEKs were never unwrapped remain protected even if their ciphertext was stolen, because the root key never left the KMS and the credential that could request unwraps is dead. For environments whose DEKs were unwrapped, the attacker holds everything needed to decrypt them offline, indefinitely. That asymmetry explains Railway’s remediation advice and a rule every user of any platform should follow: deleting a secret from a breached platform does not revoke it. Only the issuing provider can invalidate a key, and until it does, the stolen copy keeps working.

Why attackers steal AI API keys: LLMjacking and model distillation

Railway’s forensics point to a specific objective: AI provider credentials, harvested for resale into what the report calls “distributed transfer stations.” These are relay services that meter out access to frontier models, with the output ultimately used for distillation. Distillation trains a smaller model on the outputs of a more capable teacher, which takes enormous volumes of high-quality responses, and paying for them legitimately is expensive. A stolen API key shifts that bill to the victim.

The economics are not new. Sysdig’s threat research team named the pattern “LLMjacking” in May 2024, after watching attackers use credentials stolen from a vulnerable Laravel server to reach Anthropic’s Claude models through AWS Bedrock. Their tooling checked keys against ten AI services, queried logging settings to avoid detection, and referenced the OAI reverse proxy so a pool of stolen credentials could be resold without exposing the keys themselves. Sysdig calculated that a fully abused Claude 2.x deployment across four regions could cost the victim over $46,000 per day.

Black October: nine AI developer platforms breached in one month

By October 2026 the supply side had changed. Instead of collecting keys from one vulnerable server at a time, attackers went after the platforms that aggregate them. Wiz Research’s “Black October” investigation, published October 30, counted at least nine AI developer platforms disclosing incidents in a single month: Fireworks AI, Andromeda, Zeabur, Lightning AI, OpenCode, Railway, Warp, CommandCode, and Northflank. Rolling updates added more disclosures into November, including NVIDIA’s Brev and Trigger.dev.

Fireworks, OpenCode, and Warp reported indications of “agent swarm” attacks, and OpenCode and CommandCode explicitly attributed their incidents to attempted distillation. Every public report described the same objective: stored environment variables, especially those granting AI access.

What stolen AI keys cost: Wiz’s findings on Bedrock abuse

Wiz also documented what happens downstream. In two enterprise cases traced to platform breaches, the first unauthorized activity appeared within 30 minutes of the vendor-side credential access: a single InvokeModel call from a residential IP, part of a one-minute sweep testing stolen keys across multiple organizations. Days of quiet validation followed, with 14 to 120 calls a day from the same small Go client, and then the flood.

In the first case, a key permitted to use global cross-region inference profiles absorbed roughly 800,000 Bedrock calls over thirty hours. About 90 percent succeeded and only 6 percent were throttled, with the load spread across regions and concentrated on Opus-class models. In the second case, a key restricted to US profiles still absorbed roughly 1.7 million calls over thirteen hours, more than a million of them successful, and nearly all failures were quota throttles rather than denials. Together that was about 2.5 million calls and, at list prices, likely more than $100,000 of usage billed to the victims.

The traffic profiles, small cached prompts with a long tail of large inputs, resembled resold coding-agent access serving many end users. Wiz identified two overt resale operations whose infrastructure tested the stolen keys. One advertised discounted frontier-model access for USDT payments. On the other, Wiz found a page titled “distillation behaviour detection” that scored customers into risk buckets, which suggests resellers police distillation among their own buyers to avoid account suspensions. Wiz was careful to say it could not definitively tie either operation to the initial compromises or the bulk abuse.

Railway draws the same line. The misuse it observed ranged from coding agents to training pipelines and checks of task-scoring systems, consistent with credentials circulating through relay services. Whether the original intruder ran those services or only supplied them remains unestablished.

Crypto wallet drains and the Zeabur breach

AI keys were not the only thing monetized. Blockchain investigator Taylor Monahan linked the incident to roughly $371,000 drained from wallets whose private keys had been stored on Railway, across more than 500 transactions, over 200 addresses, and at least 19 EVM-compatible chains. The first theft transaction landed at 19:36 UTC on October 21. Railway has not independently confirmed those losses, so the figure is an external researcher’s estimate and not a finding of the postmortem.

Zeabur, another platform in the Black October cluster, makes a useful contrast. Its October 14 to 15 breach needed no exploit chain at all. Long-lived admin credentials sat in plaintext in a configuration repository, the cluster’s admin interface was exposed to the public internet, and the tokens never expired. The attacker read the secrets of every service on the shared cluster and ran commands inside 865 database containers. Zeabur later announced it would shut down, citing the incident.

Abliterated models and ARTEX: the AI tooling behind the attack

Railway’s report describes an opponent that behaved less like a human operator and more like a distributed process. The attacker had been creating dummy accounts since October 18, two days before the page, and automated defenses thwarted all of that early probing. During the intrusion itself, the attacker probed many potential routes in parallel while Railway hardened them in parallel, with engineers working around the clock in shifts and watching audit logs, traces, and every other available instrument 24/7.

Evidence reviewed with Wiz indicates the attacker ran abliterated models: open-weight models whose weights have been edited to suppress refusal behavior, a technique the open-source community calls abliteration. Such a model complies with offensive requests a commercial API would block. Add substantial compute and an agent harness similar to ARTEX, and the result is sustained, rapid-fire escalation attempts with no human fatigue.

ARTEX is an autonomous penetration-testing framework that a Chinese developer published to GitHub in July 2026. An operator supplies a target and an objective, a planner agent chooses each next step, worker agents run tools like nmap, curl, and Impacket, and results feed a shared graph of assets and findings. Attackers adopted it within weeks, and it went closed-source on October 8 after more than 12,000 forks. Microsoft has separately attributed a ConnectWise supply-chain compromise to the China-aligned actor Storm-0940 using ARTEX.

That context explains the tooling and says nothing about attribution. Neither Railway nor Wiz has publicly attributed the Railway intrusion to Storm-0940 or any named group, and Microsoft’s findings concern a different campaign.

One episode shaped Railway’s recovery strategy and shows what defenders face. Within an hour of patching the original isolation escape, an internal team tasked with hardening the old builder derived a working variant of the exploit by modifying the original vector. The patch had bought less than an hour of confidence. With no way to be certain every attacker-controlled account had been identified, Railway concluded the old builder could not be trusted at all.

How Railway rebuilt trust from the root CA up

Railway rotated every relevant key and secret in its tooling and infrastructure, bootstrapped new PKIs, and created a new root CA for trust. It rebuilt internal tooling from validated source so everything running could be trusted again. Any machine, service, or account suspected of attacker access was reprovisioned from scratch, and the old machines were shut down and snapshotted for forensics. Because CI was one of the attack routes, all of this happened with CI disabled. Engineers applied each rotation by hand, restarted each dependent service by hand, further restricted cloud properties, and provisioned new machines manually. Customer workloads kept running throughout.

The strategic decision was to accelerate a migration already scheduled for the week of October 26. In the weeks before the incident, Railway had built and tested a new generation of build infrastructure with a dedicated ephemeral virtual machine per build, layered on top of the original isolation so an attacker must escape two independent, industry-standard mechanisms. It also had a custom control plane separating orchestration from compute, and an entirely new storage and cache backend. Railway moved the attacker’s workspaces and all new signups onto it first. After the hardening team’s one-hour exploit variant, the gradual rollout became an immediate migration for everyone.

The operational cost of the emergency migration

The new generation shared no compute machines or storage with the old one, so Railway had to add capacity quickly, just as the disclosure triggered a wave of customers redeploying with rotated credentials. Build demand hit nearly two orders of magnitude above baseline, and the new storage architecture pushed load on the upstream storage provider to at least ten times normal. Builds were slow or failed for days until Railway and its vendors scaled capacity.

Builds returned in stages on October 22: Enterprise at 03:32 UTC, Pro at 06:29, Hobby at 21:08, and Free and Trial at 22:49. New signups were paused that evening and reopened November 2. Support tooling was fully restored October 26.

Running deployments never went down, but the build pause still hurt. A customer who revoked a leaked credential, as advised, could not redeploy with its replacement until builds returned. Railway concedes this caused downtime it had not anticipated when it chose the harder, safer migration. Where possible, the team suggested rotating credentials manually over SSH, which was itself unavailable for part of the response window.

Railway’s disclosure timing: why providers heard before customers

Many users asked why AI providers and code hosts were told before customers. Railway’s answer is defensible, though the outcome still stung.

At 09:25 UTC on October 21, with the attacker’s queries freshly parsed and eviction not yet fully validated, Railway sent GitHub, GitLab, Anthropic, OpenAI, OpenRouter, SpaceXSI, Google, and Stripe the hashes of every key it held. That let providers match and revoke potentially exposed credentials without Railway distributing live secrets. Some vendors revoked more broadly at their own discretion, which is why users far beyond the 100,000-workspace upper bound received revocation notices.

Broader customer notification waited for two reasons. Telling users to rotate secrets while the attacker might still be inside would mean watching fresh credentials get stolen again, and validated, targeted notification gives affected users the most useful head start. When the broad disclosure did come, it drove a traffic influx of one to two orders of magnitude that itself slowed recovery. Railway’s committed fix is procedural: incident runbooks will define a fixed window for ascertaining impact, after which broad notification goes out regardless. Industry debate over disclosure timing is usually abstract, so a concrete account of the trade-off is rare.

Breach costs, refunds, and Railway’s security fixes

Provider by provider, the estimated $740,000 in unauthorized spend is being unwound. AWS Bedrock and Google Gemini are refunding under existing fraud policies. OpenAI and SpaceXSI review claims case by case through a shared bridge with Railway support. Discussions with Anthropic were ongoing at publication, with Railway covering breached-key costs in the interim. OpenRouter enforces a no-refund policy, so Railway will cover its losses instead. The decrypted data also included linked-account tokens for 1,852 users, BYOK agent keys and GitHub OAuth tokens, all of which have been reset. Railway found no evidence of logins to Railway accounts using stolen credentials.

The architectural fixes map one to one onto the attack chain:

• Each build now runs on a dedicated ephemeral VM with a credential scoped to pushing one specific OCI image. The shared systems the attacker exploited are gone.
• The secrets system now lets only the secrets service reach the keys that unwrap DEKs. Every encrypt and decrypt operation is centrally audit-logged, and each calling service may decrypt only the credentials it needs, which removes the platform-wide decryption authority that sat on one orchestration disk.
• CI was rebuilt with fine-grained access control, 2FA and biometric approval in critical areas, isolated network paths, and lifecycle logging into a SIEM.
• Service accounts were re-scoped to least privilege with short-lived, just-in-time credentials wherever possible, and deprecated systems were decommissioned so dead endpoints stop being attack surface.

Railway also gained access to Anthropic’s Project Glasswing, the defensive-security program launched in April 2026 that gives critical-software organizations access to frontier models with exceptional vulnerability-discovery capabilities. U.S. Treasury approval for that access had been pending when the attack landed. A complaint is with the FBI’s IC3, a specialist forensics firm’s investigation continues, and Railway will publish indicators of compromise once law enforcement signs off.

Security lessons from the Railway breach

The lessons that carry over to other teams are:

• Treat the build job as a security boundary. It executes untrusted code while holding credentials, so its network egress and permissions deserve the same scrutiny as production, and runtime monitoring of build activity is no longer optional.
• Pin OIDC trust to exact claims, and test that unintended workloads are denied.
• Decide deliberately where decryption authority lives. Envelope encryption protects data only while the unwrap path is out of reach.
• Put a last line of defense at the AI provider, the one chokepoint every stolen key must pass through: spending caps, rate limits, anomaly alerts, and fast revocation. Wiz’s restricted-key case shows the limits of scoping, since a US-only Bedrock key still served more than a million unauthorized calls.