GLM-5.3 on Workers AI Leaked Other Users' Prompts: SGLang Fix List

Between August 27 and September 9, 2026, Cloudflare's @cf/zai-org/glm-5.3 endpoint on Workers AI returned text from other customers' requests in 155 responses, all to prompts longer than 681,280 tokens. Cloudflare and the SGLang team published coordinated write-ups on October 26. The cause was not the GLM-5.3 model. It was a deployment where the context window (1,048,576 tokens) was larger than the memory pool that backed it, plus an SGLang bug that turned a rejected request into a read of someone else's memory.

Last updated: October 27, 2026

If you run SGLang with prefill/decode disaggregation, this is an upgrade-now post. If you only call open-weight models through a hosted API, it is a short list of questions to ask your provider.

What Cloudflare says happened#

Everything in this section is from Cloudflare's incident report and is vendor-reported.

• A security researcher, Gadi Evron of Knostic, reported the issue on September 8. Cloudflare reproduced it the next morning and took the model offline within 40 minutes of declaring an incident. It restored the model on September 9.
• GLM-5.3 launched on Workers AI on August 27 with a 1,048,576-token context window, but the KV cache pool on the prefill GPUs held only 681,280 tokens. Cloudflare says it knew about the mismatch before launch, expected the engine to return an error for oversized prompts, and saw its long-context tests return successful HTTP responses.
• Cloudflare says it found no evidence of deliberate exploitation. The affected requests were consistent in timing and volume with ordinary long-context use.
• It says no other Workers AI model was affected.

The testing gap is the part to steal. Cloudflare validated status codes and performance at long context, not whether the output was coherent and belonged to the right request.

The mechanism, in plain terms#

In disaggregated serving, one pool of GPUs does prefill (reads the prompt, builds the KV cache) and a second pool does decode (generates tokens). The cache moves between them over RDMA. Pages in the cache are not zeroed between requests, so the engine must track who owns each page.

Per both write-ups, the failure went like this:

1. A prompt larger than the prefill pool is rejected by the scheduler, which swaps it for a one-token placeholder so it can drain through the batch machinery.
2. In disaggregated mode, the admission queues checked whether a request was finished, not whether it was rejected, so the rejected request went through the handoff anyway.
3. Decode had reserved pages for the full prompt length. Prefill wrote one page. Decode attended over all of them, and the rest still held other users' data.
4. The client got HTTP 200, a correct prompt_tokens count and a coherent reply.

The SGLang team's post describes a second, related bug class: when a request is aborted mid-transfer, RDMA writes already posted cannot be cancelled, so if decode frees the destination pages immediately, a late write can land inside another request's cache. They call it a distributed use-after-free, and say it is silent: no error, no log line, and a deployer cannot tell afterward whether exposure occurred. They also say the aborter cannot choose the victim and learns nothing, so the realistic impact is one-way leakage into someone else's output.

What to do Monday if you run SGLang#

This is the angle that matters for builders. The checklist below is condensed from the two vendor posts; flags and variable names are copied from them and not run by me, so check them against your version's --help.

1. Compare pool size to context length on every prefill and decode node. SGLang prints the selected KV pool size at startup and exposes it as a metric. If the pool is smaller than the context length, lower the context length. Re-check after any change to weights, quantization, parallelism or memory fraction.
2. Upgrade. Cloudflare recommends v0.5.22 or later. SGLang says v0.5.21 protects Mooncake and NIXL deployments by default. Their disclosure timeline also lists a fix for failed NIXL and MORI sends that ships in v0.5.23.
3. On v0.5.19 or v0.5.20, SGLang says to set SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE=1 on both prefill and decode, and on v0.5.20 also set SGLANG_MOONCAKE_MAX_TRANSFER_BATCH_INDICES and a 60 to 120 second SGLANG_DISAGGREGATION_DEFERRED_DECODE_KV_RELEASE_TIMEOUT. They say these do not replace upgrading.
4. Turn on corruption detection where it is supported: --disaggregation-enable-kv-checksum on both prefill and decode, and --kv-canary. Cloudflare notes neither covers every model architecture yet, so confirm in your logs that requests are actually being verified.
5. Watch the metric. After rollout, SGLang says to watch sglang:decode_deferred_kv_release_total{outcome}. A healthy deployment releases almost all holds through drained; a timeout-heavy ratio means every abort is waiting out the full hold.
6. Pair your aborts. Use a router that cancels both halves of a prefill/decode pair, and configure your load balancer to let requests run to the first token so client disconnects do not trigger cross-node aborts.

SGLang reports throughput within plus or minus 2% in its own 2xH100 tests with the new default enabled, with every confidence interval spanning zero. That is vendor-reported on one topology.

The strategic read#

Three second-order effects follow.

Shared-tenancy risk is now a line item for open-weight hosting. Part of the pitch for hosted open models is that you get frontier-adjacent quality at a fraction of the price (see our GLM-5.3 access and pricing breakdown). The price gap partly comes from packing many tenants onto one GPU. This incident is a concrete example of what that packing can cost when bookkeeping fails, and it makes dedicated or single-tenant capacity easier to justify for sensitive prompts. Run the numbers in our self-hosting break-even math with a risk premium, not just a price per million tokens.

Long context is where this bug class lives. SGLang argues the race window scales with prefill context length, so these bugs surface as deployments move to very long contexts. Providers racing to advertise million-token windows are the ones most exposed. Cloudflare's own August write-up on serving Kimi and GLM described a KV cache integrity check; in this post Cloudflare says that prototype was disabled shortly after its mid-August test because of reliability issues, and that it is now working with SGLang on a production version.

The engine is part of your supply chain. Cloudflare says another user reported the underlying SGLang issue upstream on July 6, and that Cloudflare had been flagging this class of bug since v0.5.10. If you compare engines, the question is not only throughput; our vLLM vs TGI vs SGLang comparison is the place to start, and the KV caching guide explains why the cache is shared state in the first place.

What to ask a hosted provider#

If you consume open models through someone else's endpoint, the practical questions are short:

• Is the KV pool measured against the advertised context length on every deployment, and who checks it?
• Is cache integrity checking (checksums, page tags) on, and for which models?
• Do you have a disclosure and notification process for cross-tenant exposure?
• What is the longest prompt you actually test, and do you test output coherence at that length?

For coding agents specifically, the safer baseline is to keep credentials out of prompts in the first place; see keeping secrets away from AI coding agents.

What people are actually saying#

The Hacker News thread was dominated by an argument about the title, not the bug.

• Agreement on the root cause. Commenter jacquesm argued it has nothing to do with GLM-5.3 and is a Cloudflare configuration and SGLang bookkeeping problem, and that the headline should say so. Several others agreed the title could mislead a skim reader.
• The counter-case. Cloudflare engineer kflansburg replied that the report states it was a Cloudflare misconfiguration plus an SGLang bug, and that the model is named because customers need to know which endpoint was affected. Other commenters sided with that reading: caching problems are obviously not a model problem, and the name tells you whether you were exposed.
• Praise for the write-up. One commenter liked the ownership, the clear explanation of why testing missed it, and the speed of taking the model offline.
• Skepticism about the known mismatch. Several commenters questioned why a team that knew the pool was about a third smaller than the context window shipped it; one compared it to knowingly deploying a buffer overflow risk. Another noted the figure applies to the prefill pool, so ordinary multi-turn sessions that stay cache-warm were unlikely to trigger it.
• A broader worry. Commenters asked whether other serverless providers have the same class of bug and have not disclosed it, and some described anecdotal cases of receiving unrelated responses from hosted APIs. Those are unverified anecdotes, not evidence about any provider.

FAQ#

Was the GLM-5.3 model itself at fault?#

No. Cloudflare says the cause was a pool-size misconfiguration plus an SGLang bug in how rejected requests were handled in disaggregated mode. Any model deployed with the same mismatch could have hit it.

Am I affected if I used GLM-5.3 on Workers AI?#

Cloudflare says the window was August 27 to September 9, 2026, and only prompts longer than 681,280 tokens could trigger the exposure. Check the incident report for its notification guidance and contact Cloudflare if you sent prompts that large in that window.

Which SGLang version should I run?#

Cloudflare recommends v0.5.22 or later. SGLang lists a further fix for failed NIXL and MORI sends in v0.5.23. Confirm against the current release notes before upgrading.

Does this affect single-node SGLang?#

SGLang says the single-node rejection path returned an error, and the aborted-transfer race needs prefill/decode disaggregation with a KV transfer backend. Non-disaggregated deployments are not affected by those specific issues.

Continue Reading#

• Cloudflare Runs Kimi and GLM at Scale - the August write-up on FP8 KV caches and the cache safety net
• vLLM vs TGI vs SGLang - choosing an inference engine
• KV Caching: A Practical Guide - why the cache is shared memory you have to manage
• GLM-5.3 Free and Cheap: Every Route and Price - hosted options and pricing
• Self-Hosting Open-Weight Models: Break-Even Math - when owning the GPUs pays off
• Self-Hosted vs Managed AI Gateways - control versus operations tradeoffs
• Keep Secrets Away From AI Coding Agents - reduce what a leak could expose
• Best Open-Source Models - the open-weight field, ranked
• Best Local Models - what you can self-host, by memory tier

Sources#