---
title: "GLM-5.3: Seven hundred fifty-three billion parameters, zero architectural changes, and not one capability number anyone else can check"
url: "https://chriscaruso.dev/notebooks/glm-5-3"
type: "Notebook (interactive explainer)"
site: "Caruso's Conjecture"
author: "Chris Caruso"
published: "2026-08-16"
source: "writing/notebooks/glm-5-3.md"
index: "https://chriscaruso.dev/llms.txt"
narration_minutes: 146
narration_cues: "https://chriscaruso.dev/media/narration/glm-5-3/cues"
narration_chapters: "https://chriscaruso.dev/media/narration/glm-5-3/chapters"
tags: ["Machine Intelligence", "Architecture", "Attention", "Benchmarks", "Interactive"]
---

# GLM-5.3: Seven hundred fifty-three billion parameters, zero architectural changes, and not one capability number anyone else can check

**Seven hundred fifty-three billion parameters, zero architectural changes, and not one capability number anyone else can check.**

Z.ai announced GLM-5.3 on 14 August 2026, under the title "Frontier Coding with Emergent Cyber Capabilities."[^1] The company's own documentation opens with the whole release in one sentence: GLM-5.3 "uses the same base model as GLM-5.2," and "all improvements come from post-training."[^2] The architecture did not change. The context window did not change. Not a single field of the config changed, because there is no config: no weights have been released, no technical report exists, no license has been stated, and the launch page's `huggingface` link is the literal string `"#"`. The weights are promised roughly two weeks out, pending what Z.ai calls safety evaluation and hardening.

Three weeks ago I read [Kimi K3](/notebooks/kimi-k3) end to end and called it the best-documented frontier model in existence. This notebook is that one's mirror image. GLM-5.3 is, as of this writing, the least checkable frontier release I have looked at: every capability number in circulation was produced by the vendor, several of the comparison numbers turn out to be another lab's measurements wearing the wrong label, and the model itself cannot be downloaded, bought through an API, or benchmarked by anyone outside the company.

So this notebook reads what can actually be read. The GLM-5.2 checkpoint is public, and if Z.ai's sentence is true, it *is* GLM-5.3's machine, byte for byte. I reconstructed its parameter budget from the raw safetensors headers and it reconciles against Hugging Face's count to the parameter.[^3] The launch blog renders nothing to a plain fetch, so I recovered its benchmark table and its serving contract from the compiled JavaScript bundle. Four papers, three independent inference engines, and a stack of leaderboards fill in the rest. Everything below is one of two kinds of claim. Most of it is checked against an artifact I can point at: a tensor header, a config field, a line of somebody's inference engine. The rest rests on Z.ai's word, or on my own reading of the evidence, and I flag those where they occur rather than collecting them into a caveats section at the end.

One more thing before the machine. This notebook was written in the two days after launch, deliberately, before the weights land. That window is what makes it possible to do something explainers usually cannot: state a falsifiable prediction in public, with a date on it, and be around to be wrong. It is at the end.

## The release you cannot see

A word on who this is, because it comes back near the end of this notebook. Z.ai is a Beijing lab with deep Tsinghua roots: its papers carry Tsinghua faculty as co-authors, and the training framework behind this release ships from the university's data-mining group. GLM is its flagship line, and it ships an architecture roughly once per major version and then rides it. GLM-4.6 and GLM-4.7 have byte-identical config files, the strongest form of "nothing changed" that exists. GLM-5 and GLM-5.1 differ in exactly one field, `transformers_version`, a bookkeeping string. And GLM-5.3, on the vendor's own statement, changes nothing at all from GLM-5.2. Two of the four releases in the 5.x line are architecturally invisible, and the pattern repeats one version back.

> **Interactive figure:** Four GLM releases as config columns; stepping through them highlights only the fields that changed, and two of the four light up nothing.

The one release that did change things is GLM-5.2, in June. Its diff against 5.1 has three real moves in it: the declared context window grew from 202,752 tokens to 1,048,576, the rotary base rose from one million to eight million, and a family of new config keys appeared that cut the model's sparse-attention indexer from 78 copies to 21. Everything this notebook says about GLM-5.3's machine is really a statement about that June checkpoint, carried forward on one sentence of vendor prose. The figure draws the 5.3 column dashed for that reason. There is no `zai-org/GLM-5.3` repository, so there is no config to read; the column is 5.2's values plus a claim.

The claim does have corroboration of an unusually credible kind: code written by people whose only motivation was making the thing run. No inference engine anywhere carries GLM-5.3 support, and none needs to. A comment in xLLM's model loader says it plainly: GLM-5.2 shares its `model_type`, `glm_moe_dsa`, with GLM-5.0 and 5.1, and the releases are told apart by config values alone.[^4] The entire 5.x line is one architecture discriminated by numbers in a JSON file. Fifteen independent implementations run it, and not one has a `glm5` file.

What exists of GLM-5.3, then, as of this writing: a subscription tier. The model is reachable only through the GLM Coding Plan, locked to a list of approved coding tools. The API is "coming soon." Z.ai's own release-notes page had not acknowledged the launch a day later, and its price list still stops at GLM-5.2. The announcement date at the top of this notebook is itself an inference, from CDN image timestamps, because it appears in plain text nowhere.

This is the first GLM-5.x release to separate the announcement from the weights. GLM-5, 5.1 and 5.2 all landed on Hugging Face the day they were announced, on a cadence of 51, 74 and 59 days. The two-week weights delay is new, and the stated reason for it, safety hardening, is one of the threads this notebook follows to the end. A model whose headline is emergent cyber capability is the first one its own maker held back.

## One layer, whole

Before taking anything apart, here is the entire machine, working. Everything later in this notebook is a drill-down into one stage of this figure, so give it thirty seconds and just watch a token go through.

> **Interactive figure:** One token traversing a complete GLM layer: normalize, index, attend over the keep-set, normalize again, route to nine experts, and two residual adds.

A token enters as an id and leaves as a distribution over 154,880 vocabulary entries. In between sit 78 transformer layers plus one extra. Each layer reads a running vector of 6,144 numbers, the **residual stream**, which is the token's working state as it moves up the stack, and each layer writes its own contribution back into it. The path is strict pre-norm: normalize the stream, apply attention, add the result back, normalize again, apply a feed-forward block, add that back. Two RMSNorms, two residual additions, nothing else. GLM-4.5 wrapped its layers in extra post-norms for stability; at twice the size, GLM-5.x dropped them.

The attention stage is the elaborate one, and it has two unusual organs. The stream is compressed into a small cached summary, 576 numbers per token, from which all 64 attention heads later reconstruct what they need. That is MLA, and it is why this model's memory bill is a fiftieth of what it would naively be. In front of the attention sits an *indexer*: a cheap scoring pass that reads the whole context and picks the 2,048 tokens the expensive attention is allowed to look at. That is DSA. Only 21 of the 78 layers own an indexer; the other 57 borrow the most recent selection.

The feed-forward stage comes in two kinds. Layers 0 through 2 are ordinary dense blocks. Layers 3 through 77 are mixture-of-experts layers: 256 separate small networks, of which a router chooses 8 per token, plus one shared expert that runs on everything. Those banks are where this model keeps 97.49 percent of its parameters, almost all of it asleep for any given token.

After layer 77, a final norm and an output projection produce the distribution. And then there is layer 78, which is not part of the stack at all: a complete spare layer, attention and experts and all, whose job is to draft the *next* token early so that serving can check several guesses in one pass.

Step the figure through those stages once and the proportions are the surprise: two norms, a compressed cache, a cheap scoring pass, and then a bank of experts that is most of the machine by weight and almost none of it by design effort. The rest of this notebook works through that figure stage by stage, in roughly that order.

A few oddities of the shipped checkpoint belong here before the autopsies, because each one carries information. The model contains one classical LayerNorm and one bias tensor, and they are the same object: the little norm on the indexer's 128-dimensional key. Everything else in 753 billion parameters is bias-free RMSNorm. My best reading, offered as a reading, is that the indexer is the one place where a small numerical error changes a *ranking* rather than a value: its output feeds a top-2,048 cut, and a token nudged below the line silently vanishes from attention's world. The one component whose mistakes are discrete got the one norm with a mean-subtraction and a bias.

Two norms that look like a safety feature are not one. The 2,048- and 512-wide norms inside the attention block sit on MLA's compression bottlenecks, and they are easy to mistake for QK-norm. GLM-5.x has no QK-norm anywhere. Then there are the counts, which all have to agree, and agreement is the closest thing a checkpoint offers to an audit: 79 attention blocks, 78 of them in the stack; 3 dense plus 75 expert layers; 21 indexers plus a 22nd hiding in the draft layer; 76 expert-router bias vectors, one per expert layer plus that same draft layer. Every one of those is a term in the budget the next section closes to the byte.

## Where the model lives

A word about proportion, out loud, because the shape of this notebook could otherwise mislead you. This chapter and the three after it are the architecture, and the architecture is the part of GLM-5.3 that did not change. It gets the most room anyway, for one reason: it is the part that can be *known*. The GLM-5.2 checkpoint is downloadable, so every claim about the machine can be pushed one layer deeper, checked against tensor shapes, recomputed from a script. The part that actually changed, the post-training, is one blog post with no weights, no paper and no independent number, and it gets the space that evidence supports. Length here is a map of what is knowable, not of what matters.

One convention for all four, so I can stop repeating it. Unless a sentence says otherwise, every number in them was read off the GLM-5.2 checkpoint, its config, or the source of an engine that runs it, and you can recheck any of them. There are maybe half a dozen exceptions, they are all interesting, and each one announces itself where it sits.

### A census that closes to the byte

Hugging Face reports GLM-5.2 at 753,329,940,480 parameters. I rebuilt that number from scratch: HTTP range-requests pull the JSON headers out of the safetensors shards without downloading the 1.5 TB of weights behind them, the headers list 59,585 tensors with their exact shapes, and summing the products gives 753,329,940,480. An exact match, to the parameter, and it is what licenses every other number in this chapter.[^3] The reconstruction ships with this notebook as a runnable script, and the figure below rebuilds its bars from the same per-layer constants.

> **Interactive figure:** Where 753 billion parameters sit on disk, against which ones fire for a single token. Toggling between the two views inverts the picture.

The stored view is almost comically lopsided. Routed experts are 97.49 percent of the checkpoint. Attention, the mechanism this whole model line is famous for, is 1.73 percent. The embeddings and output head together are a quarter of a percent, and the 22 indexers, the organ two chapters of this notebook revolve around, are 0.0274 percent: about 206 million parameters in a model of 753 billion.

Flip to the active view and the picture inverts. A token touches 40,298,947,584 parameters, 5.35 percent of the model. That set is every layer's attention, the 21 indexers, nine experts out of 257 in each of the 75 sparse layers, the three dense feed-forward blocks, and the output head. Within that set, attention is **31.94 percent**. On disk it was 1.73 percent. Nearly a third of the parameters this model actually reads for a token belong to the mechanism that is a fiftieth of it at rest, an eighteen-fold inversion, and it is the single most useful fact for reading everything that follows: the part of this model that barely exists on disk is a large fraction of what has to be hauled out of memory to use it.

Two details of the count settle real confusions. First, the number spread. You will see this model quoted at 744B, 750B, 753.3B and "40B active," and all of them are one model under different counting conventions. The 40,298,947,584 active figure includes the output head, which is a dense 6,144-by-154,880 matrix that runs on every token and alone is 2.36 percent of the active budget. Subtract it and you get 39,347,364,864, which is exactly the "39.35B active" Z.ai advertises. The embedding matrix, same shape, costs nothing per token, because looking up a row costs a memory read and no arithmetic.

Second, a forensic gem. Hugging Face's dtype breakdown lists exactly 19,456 parameters stored in full FP32 precision, in a checkpoint that is otherwise all BF16. That number is 256 × 76. The only FP32 tensor in the model is a per-layer vector of 256 expert-routing biases, there are 75 expert layers in the stack, and 75 does not divide 19,456. The 76th copy is inside layer 78, the draft layer, which is how you can know, from a dtype histogram alone, that the draft layer carries a complete 256-expert block of its own. Checkpoints leak structure through their bookkeeping.

One conflict belongs on the record, because it says something about reports versus artifacts. The GLM-5 technical report states, in a sentence I am quoting verbatim, that the model "reduces its layer count to 80" and totals "744B" parameters.[^5] Every shipped config says 78 layers, the tensor index contains nothing beyond layer 78's draft module, and the measured total is 753.3 billion. The checkpoint wins. A technical report is testimony; a safetensors header is evidence.

### The cache is the model

Attention has a memory problem that compounds with context. To generate token one million and one, a model must consult keys and values for the million tokens before it, and the naive ledger for this model is brutal: 78 layers × 64 heads × 512 numbers per head × 2 bytes is 5,111,808 bytes of cache per token of context. At the full window that is 4.88 tebibytes, for one conversation, before a single weight is loaded.

> **Interactive figure:** One token's key-value information being squeezed through a 576-dimension bottleneck, with a context slider driving the live byte count. The naive cache runs off the frame.

MLA is the fix, and GLM's version stores 576 numbers per token per layer. Not per head. The layer compresses each token's key-value information into a 512-dimensional latent plus one 64-dimensional rotary key shared by all 64 heads, and caches only that row. The heads' actual keys and values are reconstructed from the latent on demand, or, as the next section shows, never reconstructed at all. The arithmetic: 78 × 576 × 2 bytes is 89,856 bytes per token, plus 5,376 bytes of indexer keys on the 21 layers that keep one, for 95,232 bytes all-in. Against the naive 5,111,808 that is a 53.7× compression. A million tokens of context costs 93.0 GiB instead of 4.88 TiB. Push the figure's context slider to the right and the naive bar leaves the frame while the MLA bar is still a stripe; there is no scale on which you can draw both at a million tokens.

Two versions of that ratio circulate and both are right, so let me pin which is which. The 53.7× above is the honest serving number: all layers, indexer included. The 56.9× you will see quoted is one layer's MLA row alone, 32,768 naive numbers against 576. Use the first for anything about memory bills. The 93.0 GiB carries two caveats of its own: it assumes BF16 cache entries, which serving stacks halve to FP8 on newer hardware, and it is what production engines achieve, not what the reference `transformers` implementation does. That implementation caches the fully expanded keys and values, MHA-sized, with a to-do comment in the source admitting it. Measure GLM's memory with the reference code and you will conclude this section is wrong.

Why does GLM use MLA at all? The GLM-5 report is unusually frank about this: in their experiments, "MLA with a 576-dimension latent KV-cache cannot match the performance of GQA with 8 query groups."[^5] The compression was costing quality, and the fix was an optimizer change rather than an architectural one. Muon Split orthogonalizes each attention head's projection block independently instead of treating the fused matrix as one unit, and that closed the gap. That sentence explains more about GLM's attention than anything else Z.ai has published, including the model's oddly shaped heads: 256-dimensional values against 192-plus-64 query-key dimensions are the settlement of a quality deficit, not a clean derivation.

One config field deserves a warning label. `num_key_value_heads: 64`, equal to the attention head count, looks like it contradicts everything above. It describes the logical heads after reconstruction, not what is stored. The cache is the 576 row.

### The rotation that breaks the trick

MLA's economy rests on an algebraic move, and the move has one enemy. This section is the hardest idea in the model and the most rewarding one to watch happen, because it is a fact about matrix multiplication that you can see.

> **Interactive figure:** Two projection matrices fusing into one, a rotation wedging itself between them, and the split that rescues the fusion.

The move goes like this. An attention score is a dot product between a query and a key. In MLA neither of those exists in memory; both get built on demand from small cached vectors, each by multiplying with a fixed matrix. The query is one matrix times the query token's compressed vector. The key is another matrix times the key token's 512-number cached row.

Now write the whole score out as a single expression and look at what sits where. The two cached vectors end up on the outside, one at each end. The two matrices end up in the middle, back to back. And that middle pair is the same for every token in the sequence, because a projection matrix is a weight: it stopped changing when training finished. So you can multiply those two matrices together **once**, at load time, and from then on attend directly against the cached row, never building a single per-head key at all. That fold is called absorption, and it's the entire reason a 576-number cache can serve 64 heads. Drag the figure's first stage and watch the two blocks fuse.

Now add position. Transformers need to know where tokens are, and this family uses RoPE: rotate each query and key by an angle proportional to its position before taking the dot product. A rotation is itself a matrix, so writing the score out now gives you six things in a row instead of four. Reading from the query end: the query's cached vector, its matrix, its rotation, then the key's rotation, the key's matrix, the key's cached vector.

The two rotations meet in the middle and combine into one rotation by the *difference* of the two positions. That collapse is the elegant part of RoPE, and it's why the mechanism encodes relative position without storing anything. But look where it leaves the survivor. The combined rotation is now wedged **between** the two projection matrices, in exactly the spot where absorption needed them touching, and it is a different rotation for every pair of positions in the sequence. The figure's second stage lets you drag the two positions: the would-be fused block changes on every drag. A matrix that changes per pair cannot be precomputed once. Absorption dies, and the cache saving dies with it. Rotation does not commute with low-rank fusion.

The fix, inherited from DeepSeek and confirmed line by line in GLM's modeling code, is surgical. Split each head's 256 query-key dimensions into two parts that never mix. The large part, 192 dimensions, carries no position at all and flows through the absorbable path, where the fusion replays untouched. The small part, 64 dimensions, carries all of the position, and it cheats: its key is produced directly from the raw hidden state, bypassing the compressed latent, rotated once, and cached already-rotated. Because that rotary key never involves the up-projection matrix, there is nothing for the rotation to break. The score becomes an absorbable term plus a small positional term, summed.

The implementation details are quietly elegant. The 64 rotary dimensions are deliberately excluded from the latent's normalization, since they are not part of the latent. The rotary key is created with a head dimension of one and then broadcast to all 64 heads as a stride-0 view, an `.expand()` that copies nothing: 64 numbers per token per layer, read 64 times. Per-head rotary keys would have cost 4,096. That one broadcast is the difference between MLA surviving position and not.

## Deciding what to look at

Attention's other problem is arithmetic. Every token scoring every earlier token is a cost that grows with the square of the context, and at a million tokens the square is monstrous: run this model dense at its full window and 98.56 percent of the per-token compute is core attention. The 753 billion parameters become a rounding error against their own quadratic term. GLM's escape is a chain of three mechanisms, and the property that makes the chain worth following is that each fix promotes a new bottleneck for the next one to attack. This is the heart of the machine, and it is where the research for this notebook kept turning up things nobody has written down.

### Attention that skips

DSA, DeepSeek Sparse Attention, is a bet that most of a long context does not matter for any given token. Instead of attending to everything, each layer attends to a keep-set of 2,048 tokens, chosen fresh for every query by a cheap scoring pass called the lightning indexer. At the full window, 2,048 of 1,048,576 is 0.1953 percent. The expensive machinery reads one token in five hundred, and the whole question becomes whether the cheap machinery picks the right five-hundredth.

> **Interactive figure:** A query token scoring a ribbon of context: 32 heads, one shared key, ReLU before the sum, and the top 2,048 staying lit while the rest go dark.

The indexer's shapes are the mechanism, and one of them appears in no writeup I could find. Its query side has 32 heads of 128 dimensions, read from the same compressed latent the main attention uses. Its key side is the surprise: the key projection is a single 128-dimensional head, shared by all 32 query heads. That is why the indexer's cache costs 128 numbers per token instead of 4,096, and it is why the indexer is affordable at all. DeepSeek's paper confirms the intent in passing, and the tensor header confirms the shape directly.[^6]

There is a second economy in it, and it matters later. The indexer runs in **FP8**, both operands quantized, where the attention it gates generally does not. DeepSeek says so with visible satisfaction: because the lightning indexer "has a small number of heads and can be implemented in FP8, its computational efficiency is remarkable." Hold onto that, because this notebook is about to spend a chapter counting the indexer's share of the arithmetic, and a count of operations quietly assumes every operation costs the same.

The scoring formula repays writing out, because two details in it are easy to get wrong. Each of the 32 heads dots its query against a token's shared key. Each head's score is passed through a ReLU, then the 32 rectified scores are combined by a weighted sum whose weights the token generates from its own hidden state. The ReLU comes *before* the sum, per head, per key: a head's negative evidence is discarded before it can cancel another head's positive evidence. DeepSeek states the reason with disarming honesty: ReLU was chosen "for throughput consideration." Speed, not accuracy. The figure computes this pipeline literally, and one honesty note applies to it: GLM's learned indexer weights are not public, so the score distribution you see is synthetic. The selection arithmetic, the head structure and every count are real.

Then the detail that reframes the whole mechanism. The kernel takes the top `min(2048, context)` scores, so when the context is shorter than 2,048 tokens, the indexer selects everything, and DSA is exactly full attention. Serving stacks go further and skip the indexer below that line. Drag the figure's context slider to the left and watch sparsity switch itself off. Nothing sparse happens to your short prompts, ever; the entire apparatus is dormant until the context is long enough for it to matter. Sparsity here is a regime the model *enters*, at a context length you can point to.

GLM's DSA differs from DeepSeek's in scale and in appetite. Same keep-set of 2,048, half the indexer heads, 32 against 64, and a declared context 6.4 times longer, which stretches the same keep-set over a far larger haystack: against its own declared window DeepSeek keeps 1.25 percent, GLM keeps 0.195 percent. And one thing DeepSeek concedes in its own report deserves the emphasis it rarely gets: DSA reduces core attention from quadratic to linear, but the indexer that enables it still scores every token against every token. The quadratic term did not die. It moved into the cheap pass, and that fact sets up everything in the next section.

### Which layers get to decide

Run the numbers on DSA at a million tokens and the fix reveals its own successor. Core attention collapses by a factor of 512, and the indexer, the cheap pass, becomes 88.35 percent of the remaining per-token compute. The thing that picks what to look at now costs more than everything it was protecting combined. GLM-5.2's response is called IndexShare: stop running the indexer at every layer.

> **Interactive figure:** The 78-layer stack as a column: 21 layers stamp a shared buffer with fresh indices, 57 inherit it, and three bars fall together.

The config ships a literal 78-entry array assigning each layer `full` or `shared`. The full layers are 0, 1, 2, and then every fourth layer from 6 to 74: twenty-one in all. A full layer computes fresh indices over the whole context and attends over its selection. A shared layer has no indexer whatsoever, no weights for one in the checkpoint, and inherits the selection from the nearest full layer *before* it. The paper behind the technique implements this as a single buffer, overwritten at each full layer and read by the layers above it, which means the whole mechanism costs one buffer instead of 78, and one conditional branch.[^7] The figure shows the buffer being stamped and re-read as you walk the stack.

Two misreadings to close off immediately, because both are natural and both are wrong. Shared layers still run full sparse attention; only the *scoring* pass is skipped, never the attending. Inheritance also runs backward, to the most recent full layer, not forward or nearest-in-either-direction. Layer 0 is always full so there is always something to inherit.

There's an oddity in how that 78-entry array gets used, or rather does not. Only the reference implementation reads `indexer_types` at all. The two engines that actually serve this model in production ignore it completely and re-derive the identical pattern from two integers, a period and an offset. So the config carries an explicit, per-layer, human-readable description of which layers think for themselves, and the code path that matters never looks at it.

What one shared buffer buys turns out to be three different savings, and the figure draws all three bars. Compute: 21 indexer passes instead of 78 is 3.71× fewer, which blends to 2.82× fewer total attention-side FLOPs at a million tokens; Z.ai advertises 2.9×, and my reconstruction lands within three percent of that, so the claim checks. Weights: 57 layers carry no indexer tensors at all. And cache, the saving nobody advertises: without IndexShare every layer would bank its own 128-number index key per token, 19,968 bytes; with it, 5,376. The total cache falls from 109,824 to 95,232 bytes per token, a 13.3 percent cut in the quantity the section on decode will show is the actual bill.

One config key tells a small, sharp story about roads not taken. Both major serving engines, before applying the periodic every-fourth-layer rule, first check a field called `index_topk_pattern`, which can hold an arbitrary per-layer pattern. That is the slot where a *searched*, non-periodic pattern would go, and the search is not hypothetical: the technique's own paper ships one. GLM-5.2 sets the field to `null`. The machinery for shipping a smarter pattern exists in every engine that runs this model, and Z.ai left it empty. Whether that was laziness or confidence is the subject of the next two sections.

Put the two config fields side by side and they make a small joke about each other. One describes the layer pattern explicitly, in full, and no production engine reads it. The other is the one every engine checks first, and it's empty.

A naming note, since the literature will otherwise fork on you. Z.ai's marketing calls this IndexShare. The paper, from Z.ai and Tsinghua authors, calls it IndexCache, never uses the word IndexShare, and never mentions GLM-5.2 or 5.3.[^7] One mechanism, two names, linked only by Z.ai's own citation.

And one sentence from the GLM-5 report reads differently a release later. It calls DSA "lossless by construction," on the grounds that the indexer selects without discarding long-range dependencies, and concludes that it can therefore be applied "to all layers with no quality degradation."[^5] GLM-5.2 then took the indexer out of 57 layers of 78. The two claims are compatible if you are careful: losslessness is a property of the *selection*, and IndexShare is about reusing a selection rather than making one. Still, a lab that had just finished arguing for indexers everywhere deleted three quarters of them within four months, and the report gives no hint that was coming.

### How the indexer learns what to keep

Nothing so far explains why a 32-head scoring pass should agree with 64 heads of real attention about what matters. The answer is the best mechanism in this whole architecture, and I have not seen it explained anywhere outside the primary papers: the indexer is never trained by the language-modeling loss. It is a distilled student of the attention it replaces.

> **Interactive figure:** Two training stages: a frozen model teaching its indexer what attention looks at, then both training at once with the wire between them cut.

Stage one is a dense warm-up. Freeze the entire model except the indexer and run attention dense, no selection. At every step, take the real attention scores, sum them across heads, and normalize each query's row into a probability distribution over the context: that distribution is the target. Train the indexer, alone, to produce a distribution as close to that one as possible, scored by how many bits you would waste encoding the teacher's answer using the student's guess. That measure is the KL divergence, and it is the whole loss. The model is the teacher, the indexer is the student, and the lesson is literally *here is what I looked at, learn to predict it*. GLM-5 ran this for 2.84 billion tokens; DeepSeek's equivalent used 2.10 billion. Two labs, nearly the same modest budget, teaching the same imitation.

Stage two turns the selection on and trains everything at once, and its critical mechanism is a single graph operation. The main model now trains normally on its language loss, attending only over keep-sets. The indexer keeps training on its imitation loss, now computed only over the selected tokens. And the indexer's input is *detached* from the computation graph: gradients from the language loss cannot flow into the indexer, and the indexer's loss touches nothing but itself. Two objectives, two disjoint gradient paths, one forward pass. The figure draws the detach as a physically cut wire, and lets you reconnect it to see the failure the design prevents, each loss dragging the other's parameters toward a different goal.

Here the two labs diverge spectacularly. DeepSeek's sparse adaptation ran 943.7 billion tokens, a figure that reproduces from its published step counts. GLM-5's ran 20 billion, forty-seven times less, with the report stating flatly that they "find that it is enough."[^5] Whether it is enough is a live question this notebook returns to, and one label matters for honesty: those training numbers are GLM-5's. Nothing published describes how GLM-5.2's sparse attention was trained, at what length, or on how many tokens. The shipped config describes the machine; the training of the machine at its current scale is undocumented.

### Which layers matter, and why the answer evaporates

If 21 layers get to keep their indexers, which 21? The paper behind IndexShare ran that question to ground, and its answer is the most epistemically interesting result in this notebook, because it depends on when you ask.

> **Interactive figure:** Three layer patterns compared under two training regimes, with the winner flipping between them.

The figure has a switch for the training regime, and that switch is the finding. Ask it *training-free*, dropping indexers from a finished model with no retraining, and the choice of layers dominates everything. On GLM-5 itself, at a 200K window, a searched quarter-pattern gives up 0.4 points of long-context average, 78.4 to 78.0. A naive uniform quarter-pattern gives up 5.7, and one benchmark, GraphWalks, collapses from 92.7 to 74.9. The paper's own summary sentence: which indexer layers are retained matters far more than how many. Early layers are the most sensitive, and the stated mechanism is quotable: "their perturbations traverse the longest propagation path." An error in layer 2's selection compounds through 76 more layers; an error in layer 70's has nowhere to go.

Then ask it *training-aware*, distilling the model to expect sharing while it trains, and the entire finding evaporates. Take the paper's 30B testbed. A retrained model that keeps every indexer scores **51.0**. Drop half of them on a uniform pattern, retrain, and it scores **51.6**, above the model it was supposed to be approximating. Drop three quarters uniformly, which is the shape GLM-5.2 ships, and it scores **50.6**, four tenths below the baseline. And the searched pattern, worth seven points in the training-free world, lands *below* the all-indexer baseline too. Read that again: after retraining, the careful pattern loses to the dumb one, and throwing away half your indexers beats keeping them all. The paper's explanation is the mechanism worth keeping: retrained shared layers learn to adapt their attention to inherited indices, and the joint adaptation "eliminates the layer-specific sensitivity entirely." The model absorbs the constraint. Which layers matter is a real, measurable property of a frozen model, and it is not a property of the architecture at all.

That inversion is the strongest available defense of GLM-5.2's choices, because what GLM-5.2 ships is emphatically not the paper's answer. The searched GLM-5 pattern has 22 full layers, clustered non-periodically, with a knot at layers 38 through 42. The shipped config has 21, on a strict period-4 rule. The two disagree at 31 of 78 positions. Whatever Z.ai did, it did not copy its own paper's search result; the coherent reading, and I flag it as inference, is that Z.ai did the training-aware version, after which the search stops paying and a periodic rule is free. The paper's closing line promises that application to GLM-5, dated March; GLM-5.2 shipped in June.

Two honest touches in that paper deserve their own sentence. It reports a failed method by name, choosing the pattern by maximizing cross-layer index overlap, which came out no better than uniform, with a diagnosis of why the cheap proxy fails: overlap counts how many tokens are shared, and quality hinges on *which* ones differ. And it reports where the whole technique breaks. Training-free again, from that testbed's 50.2 baseline, a uniform quarter costs 7.2 points and a uniform *eighth* costs 14.9, collapsing to 35.3. Index reuse is not free at arbitrary ratios; the shipped quarter is near the edge of the tested-safe region.

There is also a cautionary tale in Z.ai's own prior work, and it earns a beat because it looks like a contradiction without being one. The GLM-5 report tested sliding-window attention on a 9B model with a fixed alternating pattern, and it *collapsed*: 75.28 at 128K for full attention, 6.51 for the naive interleave. A searched pattern retained 53.95. The mechanisms differ in a way that saves IndexShare from that fate: a sliding window discards distant tokens outright, while a shared layer still attends globally over another layer's selection. The reason to carry it anyway is that a fixed periodic pattern is what GLM-5.2 ships, and the published justification for it, in the DSA setting, is the training-aware result above and nothing else.

### The only long-context curve that exists

Everything above concerns cost. The question that determines whether any of it was a good idea is quality, and for sparse attention at long context, the published evidence base is one curve. This section shows it, because it deserves to be seen.

> **Interactive figure:** The one published quality curve for this attention mechanism: three variants of GLM-4.7-Flash on RULER, with the damage hiding at the right edge.

First, the context for the curve. Does DSA cost quality at all? The best evidence anywhere is DeepSeek's, and it is genuinely controlled: same base checkpoint, same post-training pipeline and data, with sparse attention as the sole deliberate variable. Across its 14 benchmarks the sparse model wins 7, loses 6 and ties 1, with the losses concentrated where the model generated fewer reasoning tokens, a confound DeepSeek names itself. And GLM's own 128K ablation on the GLM-5 base shows the same rough wash: sparse wins two of four tasks, one by 6.3 points. At the contexts these tests ran, roughly neutral is a fair verdict, and neutral-at-512-fold-savings is the entire proposition.

The curve that complicates it is Z.ai's, from the GLM-5 report: GLM-4.7-Flash on RULER at six lengths from 4K to 128K, in three variants.[^5] The dense baseline drifts down as context grows, ending at 79.21. The fully trained sparse model tracks it within a point everywhere, actually beating it at three of the six lengths, and gives up 0.35 at 128K. Sparsity, trained through, costs almost nothing on this instrument.

The revealing row is the middle one: warm-up only, the frozen model with a distilled indexer bolted on, stage one without stage two. From 4K to 16K it is indistinguishable from the baseline, within half a point. At 32K it slips three points. At 128K it has lost 7.86. Sweep the figure's context control and watch the line peel away only at the right edge. The damage from imperfect selection is invisible in the regime where most testing happens and concentrates where the keep-ratio tightens, and joint training is what repairs it. At 128K the indexer keeps 1.56 percent of the context. That is the deepest point on the only published curve.

GLM-5.2's headline window sits eight times past it, at a keep-ratio eight times tighter, and here the evidence simply stops. I looked hard for anything at longer range, and the full census is: no RULER, MRCR, LongBench, HELMET or long-context reasoning score exists for GLM-5.2 or 5.1, from any party, at any length. The deepest quality-suite evaluation of any GLM sparse-attention model is that 200K window from the pattern paper, on GLM-5, before IndexShare existed. The deepest independent probe of any kind is a hobbyist's, not a lab's: a reproducible 4× DGX Spark recipe that retrieved a needle verbatim from a 249,945-token prompt, with a companion claim of stable decode to 638,976 tokens.[^8] A single needle is a smoke test, not a benchmark; it saturates on models that fail RULER badly, and the same caveat applies to the only other third-party long-context result I could find, 15 of 15 needle retrievals to about 118,000 tokens in SGLang's own validation notes. And at the advertised million, there is nothing. Not from Z.ai, not from anyone, on any GLM model, ever. The headline feature of this machine is unaudited, and the sections that follow keep having to say so.

## The bulk

Attention got two chapters because attention is where the ideas are. The parameters are somewhere else. This chapter is the 97.49 percent, plus the two subsystems that ride alongside it: the rotations that carry position, and the spare layer that guesses ahead.

### Two hundred fifty-six experts, eight chosen

Every layer from 3 to 77 replaces its feed-forward block with a bank of 256 experts, each a small three-matrix network a third as wide as the residual stream. A router picks 8 per token, one shared expert runs on every token regardless, and the other 248 sit idle. Multiply it out and those 75 mixture-of-experts blocks come to 727,724,870,400 parameters between them, 96.6 percent of the checkpoint. Counting only the routed experts, and counting the draft layer's bank as well, gives the 97.49 percent from two chapters back. Either way, everything else in this notebook is a story about the last few percent.

> **Interactive figure:** 256 expert scores computed live: sigmoid, plus a draggable bias, top-8, then the gate weights gathered from the unbiased scores. The winners change; their heights do not.

An expert here is nothing exotic. It's the same feed-forward block a plain transformer puts after attention: project the token up, bend it through a nonlinearity, project it back down. GLM's version is a bias-free SwiGLU with three matrices, and its inner width is 2,048 against a residual stream of 6,144. That makes one expert a twelfth the size of the four-times-wider block a classic transformer would use. Fine-grained, in this literature, means cutting the feed-forward into many small pieces so the router has real choices, instead of a few fat ones where picking two of eight barely counts as a decision.

The routing is where the design lives, and the order of operations is the trick.[^9] The router is one matrix, 256 rows of 6,144 numbers, run in fp32 and carrying no bias of its own. It emits one score per expert. Those scores go through a **sigmoid**, applied to each expert independently, so they are not a probability distribution and do not sum to anything in particular. Then a per-expert **bias vector** is added, 256 numbers held separately in the checkpoint. The top 8 of the biased scores win. And then the bias is thrown away: the weights that actually scale the winners' outputs are gathered from the *unbiased* sigmoid scores, renormalized to sum to one, and multiplied by 2.5.

That discard is the step reimplementations get wrong, and it's the entire definition of auxiliary-loss-free load balancing. **The bias moves which experts win. It never moves how much a winner contributes.**

The reason it has to work that way repays a paragraph, because the alternative is what everyone used to do. A router left alone collapses. Whichever experts start out slightly popular receive more tokens, more tokens mean more gradient, more gradient makes them better, and better makes them more popular. Left alone for long enough, a 256-way bank ends up doing the work of a handful, and the rest is dead weight you paid to train. The classical fix bolts a balancing penalty onto the loss, which then competes with the language objective for the same gradient, and the model pays for balance in quality. The bias is a controller instead. It sits outside the gradient machinery, nudged by hand between training steps, pushing over-subscribed experts down and starved ones up until load flattens. Because it's stripped off before the gate weights are read, none of that pressure leaks into the model's arithmetic. Drag the bias in the figure: the set of lit experts shifts, and the heights of the bars stay pinned. The two rails underneath say the same thing twice over. The upper one is what the bias chose; the lower one is what plain top-8 would have chosen from the same scores. Drag the hump along the bank and the upper rail follows it while the lower rail does not move at all.

One honest gap. Those bias tensors are physically in the checkpoint, 76 of them, shape 256, registered as buffers outside the gradient and pinned to fp32. The *rule that updates them* is not published. I scanned the GLM-5 report end to end for any description of a bias update, a rate, or a sequence-level balance loss, and there is none; MoE routing is close to absent from its architecture section. GLM-4.5's paper gives a rate of 0.001 for the first 15 trillion tokens and zero after, and it's plausible that carried over, but plausible is all it is. Importing DeepSeek's number here would be worse, because that's a different lab's hyperparameter for a different model.

Two structural consequences fall out of the ordering. First, because renormalization pins the eight weights to sum to one, the multiply by 2.5 makes the routed branch's total gate mass **exactly 2.5 for every token, always**. The shared expert sits outside all of it with an implicit weight of one. So the amplitude ratio between the routed bank and the shared expert is 2.5 to 1 by construction, identical for every token the model has ever seen. Swap the renormalization and the multiply around, so the scaling happens before the sum is pinned, and 2.5 becomes a soft scale on a wandering total. You have built a different model.

Second, and this one is invisible unless you read the config's small numbers. GLM inherits DeepSeek's `noaux_tc` routing code, which implements *group-limited* routing: experts are partitioned into groups, a token may only draw from a few groups, and the rest are masked to negative infinity before the top-k. DeepSeek-V3.2 sets 8 groups with 4 selected, capping each token at 128 candidate experts and, once the 256 experts are spread across a cluster, at most four machines. GLM sets **one group, one selected**. The mask is all ones. The masking operation runs and masks nothing. GLM-5.x performs plain global top-8 over all 256 experts, and does no node-limited routing at all, which means it gave up the bound on cross-machine traffic in exchange for unconstrained routing freedom, and leans on a fast library for shuffling tokens between machines to absorb the difference. The figure's second panel draws both configurations side by side; the code path is identical and the config decides whether it does anything.

Then there's a result that inverts the usual intuition, and I had to check it twice. A GLM MoE layer is computationally **wider** than a GLM dense layer. Nine experts fire per token, eight routed plus the shared one, at 2,048 each, for 18,432 of active feed-forward width, three times the residual stream. The dense layers at the bottom of the stack are 12,288 wide, only two times. Everyone repeats that MoE layers cost about what a dense layer costs, and here they cost 1.5 times more. GLM did not buy sparsity to make layers cheaper. It bought capacity, at a 50 percent *increase* in per-token feed-forward work, and paid for that by making the three dense layers unusually narrow. The total routed capacity sitting in one layer is 524,288 units of width, eighty-five times the residual stream, of which any given token touches a thirty-second.

### Rotations, and the sixty-four dimensions that carry them

Attention compares tokens by dot product, and a dot product doesn't know about order. Something has to tell the model that a word came before another word, and in this family the answer is a rotation.

> **Interactive figure:** A head's 256 dimensions with 32 rotating pairs among them, real angles at real positions, and a count of how many pairs still outlast the whole context as the base moves from one million to eight.

RoPE works by pairing up dimensions and treating each pair as a point on a plane. Before the dot product, each pair is rotated by an angle proportional to the token's position. Pair number zero rotates fast, a full turn every handful of tokens. The last pair rotates slowly enough that it barely moves across the whole context. Think of a row of clock hands at wildly different speeds, all reading the same time: the fast hands resolve fine distinctions between nearby positions, the slow hands keep coarse track of far-apart ones, and what the rotation contributes to a dot product turns on the *difference* of the two positions and nothing else. Relative position falls out of absolute rotation, for free, with no extra parameters.

GLM rotates 64 of each head's 256 query-key dimensions and leaves the other 192 alone. The absorption argument two chapters back explained why: the unrotated 192 flow through the absorbable path that makes the small cache possible, and only the 64 carry position. Thirty-two pairs, then, in a head of 256, for a partial-rotary factor of one quarter.

There's a trap in the config here, and it is an easy one to fall into. GLM-5.2's `config.json` contains `"head_dim": 192`, and GLM-5 and 5.1 both say `64`. Neither value is used for anything. The config class overwrites the field in its constructor, unconditionally, pointing it at the rotary width, because the rotary table is inherited from a Llama implementation that reads `config.head_dim` to decide how many frequencies to build. So the rotary table is 64 dimensions wide in all three models, and the actual attention head is 256. Any description that quotes `head_dim: 192` as the head size is reading a field the model ignores.

GLM-5 and 5.1 shipped a base of one million and a window of 202,752. GLM-5.2 raised the base to **eight million** and the window to 1,048,576.[^10] The context grew 5.17 times; the base grew 8 times. And that's the whole long-context recipe. The `rope_type` field says `"default"`, which means no YaRN, no NTK scaling, no interpolation, no extension technique of any kind.

Why eight, though? The angle for pair *i* at position *p* is *p* divided by the base raised to a power that grows with *i*, so the base is the only knob in the scheme, and it does not act evenly. The fastest pair's exponent is zero, so raising the base does nothing to it whatsoever: pair zero turns once every 6.3 tokens under either base. Everything slower stretches, and stretches more the slower it already was. Going from one million to eight multiplies the slowest pair's wavelength by 7.5 and the middle of the range by about 2.8.

The slow end is where it matters. A pair whose wavelength is shorter than the context repeats inside it, so two positions a full wavelength apart get an identical rotation and that pair cannot tell them apart on its own. Only pairs whose wavelength exceeds the whole context stay unambiguous across it, and there are never many.

Count them and the decision falls out. At the old 202,752 window with a base of one million, **7 of the 32 pairs** had a wavelength longer than the context. Stretch the window to 1,048,576 and leave the base alone, and that falls to **4**. Raise the base to eight million and it returns to **7**. On this reading GLM-5.2 did not choose 8 to reach a million tokens. It chose 8 to arrive at a million tokens carrying the same long-range budget the model had at 200K. Switch the figure's base control and watch the count come back. Watch *how*, too: the spans stretch unevenly, the slow pairs shoot outward, and pair 0 does not move at all.

That last step is my arithmetic rather than Z.ai's: the config states the base and says nothing about why. It is at least not a fitted coincidence. The threshold has a closed form, and it works out to 4,838,354; eight million is a round number sitting 65 percent past it.

DeepSeek-V3.2, which shares nearly every other architectural choice with this model, does the exact opposite. Its base is **ten thousand**, the original transformer's value, sized for a 4,096-token window, rescued out to 163,840 by YaRN at a factor of 40. Two labs, one architecture, and two philosophically opposed answers to position: GLM retrained the frequencies it wanted, DeepSeek stretched the ones it had. One downstream consequence, since YaRN normally adjusts the attention temperature to compensate for its own stretching: with `rope_type` at default that adjustment is one, so the number GLM divides its attention scores by before the softmax is the textbook inverse square root of 256, computed across all 256 concatenated dimensions.

Two layout traps close the section, and both fail silently. GLM pairs dimensions **interleaved**, taking neighbors as pairs, while DeepSeek pairs each dimension with the one 32 slots away. The two produce identical results given matching kernels and silently wrong output given mismatched ones, because nothing crashes when you rotate the wrong pairs. And inside a single GLM layer the two attention paths disagree with each other about concatenation order: the main attention builds its head as unrotated-then-rotated, and the indexer builds its own as rotated-then-unrotated. Opposite orders, same layer, both correct, both load-bearing.

### Saying more than one word at a time

The model has 79 attention blocks and uses 78 of them. The last one is a complete spare layer whose only job is to guess.

> **Interactive figure:** The 79th layer folded back on itself: the residual stream and the token just emitted fused into one state, run recurrently, with the draft-step index reuse drawn as a second axis of sharing.

Generating text one token at a time is a bandwidth problem before it's a compute problem. To produce a single token, a serving stack reads every active weight out of memory, roughly 40 billion parameters' worth, does a comparatively trivial amount of arithmetic with them, and throws the read away. Speculative decoding is the standard escape: have something cheap guess the next few tokens, then run the real model **once** over the whole guess and check all of them in parallel. Verification costs about as much as generating one token, because the expensive part was the weight read and you only did it once. Every guess that survives is a token you got for free.

GLM's guesser is multi-token prediction, and it lives at `model.layers.78`. The checkpoint stores it as one layer, and it's a serious one: two norms, a fusion matrix of 6,144 by 12,288, a full attention block, its own indexer, and a complete 256-expert mixture-of-experts block. Nearly ten billion parameters of spare machine. It shares the embedding table and the output head with the main model, so it speaks the same vocabulary.

The fusion matrix is the mechanism. It takes two things and makes one: the residual stream at the end of the main stack, which is *where the model was*, and the embedding of the token just emitted, which is *what it just said*. Concatenate those, project back down to 6,144, and you have a state that knows both. Run the layer on it and you get a guess at the next token. Feed that guess's embedding back in with the layer's own output and you get a guess at the token after. The layer is applied recurrently, so drafting deeper costs more compute and not one extra parameter.

Four numbers describe this layer in different sources and they look like a contradiction. The config says one. The GLM-5 report says three. The report's evaluation says four. The vLLM serving recipe says five. All four are correct, and together they make a ladder: **one layer stored, unrolled three deep during training, evaluated four deep, served five deep.** Training with the parameters shared across three unrolled steps is what teaches one layer to be recursive at all. GLM reports an accept length of 2.76 at four speculative steps, against DeepSeek-V3.2's 2.55, meaning that on average a little under three tokens come out of each real forward pass.

There's a second kind of reuse hiding in this layer, and it's the one piece of the architecture that no vendor documentation describes. A config flag called `index_share_for_mtp_iteration` sits in GLM-5.2's file, absent from the reference implementation's own config class, surviving only as an unrecognized keyword argument. I looked for it in the serving stacks and initially concluded it wasn't implemented anywhere. That was simply wrong; it's implemented in both major engines.[^11] Draft step zero computes its own sparse-attention selection over the context, and steps one and onward reuse it, skipping the indexer on every guess after the first.

Keep that separate from IndexShare, because merging them is the natural mistake. IndexShare reuses one selection **across layers**, going up the stack. This reuses one selection **across draft steps**, going forward in time. Two independent axes on which this model refuses to recompute what it thinks it already knows, and the figure draws them as two axes for that reason.

The payoff is a count that the parameter census could not previously explain. If the draft layer inherited the backbone's skip pattern it would be a shared layer, and a shared layer reads a buffer written by the layer below it. The draft layer has no layer below it inside its own pass, so it would read a buffer nobody wrote. The engines therefore exclude it from the pattern and give it a full indexer of its own, always. That's why the checkpoint carries indexer tensors on 22 layers when the config's array names 21.

## The arithmetic

Three separate sparsities are running in this model at once, and they compose. This chapter multiplies them out, then spends its second half taking most of the result back, because the number everyone will quote is true in a regime nobody has measured and false in the one the model actually shipped in.

Everything here is per token and at prefill, which is the pass that reads your prompt rather than the one that writes the answer. Those two are bound by different things, and the last section of this chapter is about the one that sets the price. The numbers come out of a script that ships with this notebook.[^12] Where a figure appears below, it computes its own bars from the per-layer constants.

### Three sparsities, multiplied

Start with the model that GLM-5.2 would be if you deleted every clever thing in it: full attention over the whole context, all 257 experts firing, no indexer. At a million tokens of context that hypothetical model costs 6.842 trillion floating-point operations per token.

> **Interactive figure:** A waterfall from the dense model down to the shipped one, with each stage switchable, and the composition bands rearranging as the total shrinks.

A word on the unit, since "FLOPs per token" is doing a lot of work here. It counts arithmetic operations, two per multiply-accumulate as everyone in this literature counts them, needed to push one token through the network once. It's a proxy for cost, and it ignores everything about how fast memory is, which matters enormously and gets its own section in a moment. As a way of comparing architectural choices against each other it's the cleanest instrument available, because it doesn't depend on whose GPU you ran on.

Switch the three sparsities on one at a time in the figure and the bar does not shrink evenly. One of them takes almost everything, and what makes the sequence worth watching is not the total at the end but the way the colored bands inside the bar keep changing places.

The first sparsity is the mixture of experts, and it's the one everybody already knows about. Eight of 256 routed experts fire instead of all of them, so the feed-forward banks cost a thirty-second of what they would dense, and the total falls to **5.438 trillion**. A thirty-two-fold cut in the banks buys a 1.26-fold cut in the bill.

That gap is the first thing this chapter has to teach. The saving everyone talks about is nearly invisible here, because at a million tokens of context the expert banks are 0.94 percent of the budget and the model is not spending its time on its weights at all. Every number after this one keeps 8 of 256 firing, which is why the dense baseline you will see quoted elsewhere, including by me until I checked, is the 5.438 and not the 6.842.

Look at where that attention-dense model's 5.438 trillion operations actually go: **98.56 percent of them are core attention.** Every token scoring every earlier token, at a million tokens, buries 753 billion parameters under their own quadratic term. The weights are a rounding error. This is the fact that turns long context from a memory problem into an architecture problem, and it's the reason the next two sparsities exist.

The second sparsity is DSA. Attention reads 2,048 tokens instead of 1,048,576, so core attention falls by a factor of exactly 512. Not approximately: 1,048,576 divided by 2,048 is 512 with nothing left over. Total per-token cost drops to 760 billion operations, a 7.16-times saving. The composition rearranges with it. Core attention, which was 98.56 percent, is now 1.38 percent. The indexer, which didn't exist in the dense model, is **88.35 percent**.

Read that twice, because it's the shape of the whole design. The mechanism that decides what to look at now costs more than everything it was protecting, combined and multiplied by seven. DSA did not remove the quadratic term. Every token still scores every earlier token; it just does it with 32 cheap heads against one shared key instead of 64 expensive heads against 64 keys. DeepSeek says so plainly in its own report, and the point tends to get lost in the excitement about the 512.

The third sparsity is IndexShare, and it exists to attack that. Twenty-one indexer passes instead of 78 cuts the indexer's cost 3.71 times, and the total falls to 269 billion operations per token. Against the 5.438 baseline that's **20.19 times cheaper**. Z.ai advertises 2.9 times fewer total FLOPs from IndexShare alone; my reconstruction lands at 2.82 blended, inside three percent, so the claim checks out. And the indexer, after being cut by three quarters, is *still* **67.13 percent** of the budget.

That's the spine, and it fits in one line before the next section takes it apart: at a million tokens, GLM-5.3 does not spend its time on its weights. It spends its time deciding what to look at. Each fix works, and each fix promotes the next bottleneck.

One qualifier belongs on that number, and on every share in this chapter, and it cuts against the story I am telling. These are counts of operations, and operations are not all priced alike. The indexer runs in FP8; the attention it gates generally does not. So a count overstates the indexer's share of the *clock*, by up to a factor of two on hardware with FP8 units. Price the indexer's operations at half and its 67.13 percent becomes **50.52 percent** of the time, which is still a majority and barely. The headline says the indexer costs twice everything else put together. The honest version says it costs about the same as everything else put together, and that is a different sentence.

One number in circulation does not belong on this axis. The GLM-5 report says DSA "reduces the attention computation by roughly 1.5-2× for long sequences," which is a third of the smallest defensible FLOP ratio here. I don't think it's a contradiction; I think it's a wall-clock measurement rather than an operation count. The precision point above is one reason a real speedup lands far below the arithmetic, and there are four more. Kernels at these shapes are memory-bound, so removing arithmetic buys less than proportionally. The indexer's own cost is inside the measurement. Packed training batches average about half the nominal sequence length. And every prefix shorter than 2,048 tokens gets no sparsity at all, because there's nothing to prune. That reading is my inference and the report doesn't state it. Treat 1.5 to 2 as an observed speedup and 5 to 20 as a FLOP ratio, and never offer one as evidence for the other.

### Where the bottleneck actually sits

Everything in the last section was computed at a million tokens of context. That was a choice, and it turns out to be the choice that decides the answer.

> **Interactive figure:** Five bands of per-token cost, summing to 100 percent, rearranging live as you drag context length across three orders of magnitude. Both crossover points fall inside the shaded region where nobody has ever measured anything.

Run the same arithmetic at 202,752 tokens, the window GLM-5 and GLM-5.1 actually shipped, and the ordering is not merely different. It's inverted:

```figure:sheet?key=composition-inversion
label: The same five bands at two window sizes
head: Component | at 202,752 | at 1,048,576
align: left | right | right
row: MoE feed-forward | **41.16%** | 18.92%
row: indexer | 28.49% | **67.13%**
row: attention projections | 20.79% | 9.56%
row: core attention | 8.46% | 3.89%
row: dense feed-forward | 1.10% | 0.50%
note: Shares of operations, with the precision caveat from the last section still attached to every row. The two largest bands swap places.
```

At 200K this is a feed-forward-dominated model. The experts are the bill, the indexer is a quarter of it, and the entire "the indexer is the bottleneck" story that the last section built has not happened yet. Drag the figure's slider and watch the bands trade places.

Solving for where they cross gives two thresholds, and both are solved numerically in the same script as everything else. The indexer overtakes the expert banks at **293,941 tokens**, where both sit at 36.53 percent. It becomes an outright majority of the budget at **512,337 tokens**.

So the bottleneck is not a property of this architecture. It's a property of where you run it. That single sentence is the most important thing in this chapter, and I nearly published the chapter without it.

Now put the thresholds next to the evidence. The deepest quality-suite evaluation of any GLM sparse-attention model is a 200K window, from the IndexCache paper, measured on GLM-5 before IndexShare existed. The deepest independent probe of any kind is a hobbyist's needle test at 249,945 tokens. **Both crossover points fall beyond both.** Everything to the right of about 250,000 tokens on that figure is territory where no public measurement of this model family exists, from Z.ai or from anyone else, and the figure shades it for that reason.

The uncomfortable conclusion, stated plainly: the regime in which this architecture's story is most dramatic is a regime that only exists in GLM-5.2 and later, and that nobody has ever published a measurement in. The 20.19 times headline is true. The indexer-is-the-bottleneck framing is true as an operation count, and a coin flip once you price FP8 in. All of it lives in unmeasured territory. I find the caveated version more interesting than the clean one, because it says something real about how far ahead of measurement this whole field is running.

### None of this applies at decode

A FLOP analysis will never tell you the next part, and the next part is what sets the price.

> **Interactive figure:** An operating point that cannot reach the ridge. Two sliders move it, and one of them drags it back as fast as the other pushes it forward.

A language model does two very different things. **Prefill** reads your prompt: thousands of tokens arrive at once, the GPU has an enormous amount of parallel arithmetic to do, and the limit is how fast it can compute. Everything above this section is about prefill. **Decode** writes the answer: one token at a time, each depending on the last, with no parallelism to be had inside a single sequence. For every single token generated, the machine must read every active weight out of memory and the entire conversation's key-value cache along with it. Then it does a few billion operations, which for a modern GPU is nothing, and throws the read away.

The ratio between those two quantities has a name. **Arithmetic intensity** is operations performed per byte moved, and every accelerator has a break-even value where its compute throughput and its memory bandwidth are balanced. An H100 does 989.5 trillion operations per second and moves 3.35 terabytes per second, so its break-even is **295.4 operations per byte**. Above that number you're limited by math. Below it you're limited by memory, and buying a faster chip does nothing unless the memory got faster too.

GLM-5.3 at decode:

```figure:sheet?key=decode-intensity&placement=wide
label: Five operating points, all of them memory-bound
head: batch | context | KV cache bytes | KV vs weights | intensity
align: right | right | right | right | right
row: 1 | 8,192 | 0.78 GB | 0.02x | **1.92**
row: 1 | 131,072 | 12.5 GB | 0.30x | **1.50**
row: 32 | 8,192 | 25.0 GB | 0.61x | **38.95**
row: 32 | 131,072 | 399.4 GB | **9.68x** | **5.85**
row: 256 | 8,192 | 199.7 GB | 4.84x | **85.63**
note: The H100 breaks even at 295.4 operations per byte. Nothing here comes close, and the row that comes closest is the one carrying the least context.
```

Every entry is far under 295.4. The best case in the table, batch 256 at short context, reaches 85.6, which is 3.45 times below the break-even. The obvious move is to raise the batch size, since serving more users at once amortizes one weight read across all of them. Ignoring the cache, you'd need a batch of 152 to reach the ridge. But the cache grows linearly with batch, so every user you add to chase compute brings along the traffic that keeps you underneath. Drag the figure's batch slider all the way right and watch the dot refuse to arrive. **There is no batch size that makes this model compute-bound at decode.**

Three things follow. Two of them send you back to earlier chapters, and the middle one is new.

First, the key-value cache stops being a footnote and becomes the bill. At batch 32 with 131,072 tokens of context, the cache is 9.68 times the size of the weights, which means 90.6 percent of everything the memory system moves is cache. I presented MLA's 53.7-fold compression and IndexShare's 13.3 percent cut as *storage* savings, which is how they're usually described and which undersells them. They are bandwidth savings, and bandwidth is the scarce thing.

Second, FP8 is not the two-times weight-read saving it's sold as. Quantizing the model to eight-bit halves most of it, but the output head stays in higher precision, along with the router gates, the norms and the entire draft layer. The output head alone is 951 million parameters of dense work on every token. The real speedup on the weight read is **1.954**, and the gap between that and 2.0 is almost all one matrix.

Third, this changes what the draft layer is for. I introduced multi-token prediction as a latency trick, the standard framing. In bandwidth terms it's better than that: a draft token that survives verification is a token that **never required its own weight read**. At batch 1 and 8K of context, MTP takes the effective bytes moved per output token from 42.0 gigabytes down to 15.2. It's a 2.76-times cut in the dominant cost term, which is to say that multi-token prediction is a bandwidth optimization wearing a latency costume.

The scope here is single-device arithmetic. No tensor-parallel communication, no attention operations counted in the intensity figure, and H100 numbers because they're public. All three omissions push the intensity *down*, so the conclusion survives them: decode is bandwidth-bound by a factor of three and a half at its very best.

## What it costs, and what you get

The architecture chapters exist to explain a pricing fact. A model that reads a fiftieth of the memory and a twentieth of the arithmetic of its dense equivalent is a model you can serve cheaply, and cheap serving is what pays for the thing GLM-5.3 actually is. Reinforcement learning on long-horizon agentic tasks means generating enormous quantities of tokens that get thrown away, over and over, for weeks. The machine is the reason the training run was affordable.

So this chapter follows the money, and it turns up three things you would want to know before the release itself: what a token costs, what precision you were actually sold, and what the model literally receives when you talk to it.

### One name, thirty-three prices

Z.ai's own list price for GLM-5.2 is $1.40 per million input tokens and $4.40 per million output. Anthropic's Fable 5 scores higher on Artificial Analysis's intelligence index, 62 against 53, and running AA's full evaluation suite cost **$734.50** on GLM-5.2 against **$5,455.22** on Fable 5. That's the pitch in one comparison: most of the capability at an eighth of the bill.

> **Interactive figure:** Thirty-three real endpoints for one model name, plotted on context against price and colored by declared precision. Drag what you demand of a provider and the ones that cannot serve it fall into a gutter along the bottom.

The more interesting number is what happens when you go to buy it. I pulled OpenRouter's endpoint list for the single model name `z-ai/glm-5.2` and got **33 distinct served endpoints**, and they do not agree with each other about anything.[^13]

Input price runs from $0.3248 to $2.3100, a spread of **7.1 times**, for the same weights. Context windows run from 96,890 tokens to 1,048,576, a spread of **10.8 times**, and only 22 of the 33 serve the full million. A reader who chose this model for its headline context window has a two-in-three chance of getting it, and no interface tells them which they landed on. Drag the figure's handle to whatever you actually need, a floor on context or a ceiling on price, and every endpoint that cannot meet it drops into the gutter along the bottom, with a running count. The million-token line is the one to watch: eleven endpoints sit under it.

The third surprise is that price does not track precision. One provider serves four-bit quantized weights at Z.ai's exact list price. Another serves eight-bit at a quarter of that price. Seven endpoints decline to say what precision they run at all, and two of those charge $2.10. Meanwhile Z.ai's own endpoint sits near the top of its own market: eighteen of the thirty-three undercut the lab that trained the model, the cheapest by 4.3 times.

One correction to my own reading of that list, since the number invites a wrong conclusion. Five providers appear twice at different prices, presumably as latency or throughput tiers, so 33 endpoints is not 33 operators. The spread is real; the count of independent sellers is smaller.

And then the part that grays the whole panel out. **None of this exists for GLM-5.3.** OpenRouter lists thirteen GLM models and the newest slug is `z-ai/glm-5.2`. Z.ai's own price list stops at GLM-5.2. The only way to reach GLM-5.3 as I write this is a subscription called the GLM Coding Plan, whose credits are metered by a formula that bills output tokens at **24 times** the rate of input, and which is locked to a list of approved coding tools, with no general API access. Two different Z.ai documents describe two different peak-hour multipliers for that plan, one saying off-peak costs half and the other saying peak costs triple, and GLM-5.3 is named explicitly in the harsher of the two.

The plan also ships a vision service backed by GLM-4.6V, an older and much smaller model. That's the vendor conceding in its own product that the flagship cannot see. Every image in your repository is handled by something else.

### What you actually bought

Precision is sold as a billing detail. It is a capability variable, and the numbers are not subtle.

> **Interactive figure:** The same model at seven precisions, against what fits on real hardware and what each one does to top-1 agreement.

In its native BF16 the checkpoint is about 1.5 terabytes, which is more memory than any single machine you can rent has. At native FP8 it fits on one eight-GPU H200 node, though not at the full million tokens of context; that wants a B200 node. Below that the community quantizations take over: about 465 gigabytes if you compress only the experts to four-bit, 372 to 475 for a general four-bit build, 245 at two-bit, and 223 at one-bit. A 753-billion-parameter model on a machine you could put under a desk. Step the figure down that ladder and watch the second track: the memory requirement falls off a cliff, and somewhere below four bits the model's agreement with itself starts falling too.

The cost of that compression has been measured, and the instrument is top-1 agreement: how often the quantized model's most likely next token matches the original's. Four-bit and five-bit builds come out close enough to call lossless. Two-bit agrees about 82 percent of the time. One-bit agrees **about 76 percent** of the time, which means roughly one token in four is a different token, and errors in autoregressive generation compound rather than average out.[^14]

Set that beside the seven endpoints that decline to declare their precision, and the picture is uncomfortable. You can buy a model whose identity is a name, at a price that doesn't tell you what you got, and the difference between the best and worst version of it is larger than most of the model-to-model gaps in the benchmark table this notebook is about to audit.

For scale at the other end: Z.ai's own API measures around 127 tokens per second, an eight-GPU H20 node with a speculator manages 70-plus, and the honest floor is a three-bit build on a single workstation card spilling to an NVMe drive at **0.7 tokens per second**, which the person who measured it described as not remotely usable interactively. Same weights, a factor of 180 between the ends.

### What the model actually sees

Every conversation you have with this model is a single flat string, assembled by a template you never see, and GLM's template does something unusual with your history.

> **Interactive figure:** A four-turn conversation rendered in the real template, with prior-turn reasoning vanishing as the turns advance and the effort parameter landing somewhere other than where it was sent.

The frame is `[gMASK]<sop>`, followed by a system message carrying a line that reads `Reasoning Effort:` and a level. Reasoning goes in a `<think>` block. Tool results come back under a marker called `<|observation|>`, and tool calls use a bespoke XML shape where everyone else emits JSON. Tools marked for deferred loading are held out of the prompt and injected mid-conversation when they become relevant, which is a thoughtful piece of context management.

The unusual part is what happens to the model's own thinking. **GLM strips prior-turn reasoning by default.** Turn three does not get to see how the model reasoned in turn two; only the conclusion survives. Kimi K3 does the opposite and preserves it. Both choices are defensible and they encode opposite bets: GLM is betting that stale reasoning is a contaminant and the conclusion is the artifact, Moonshot is betting that a chain of thought is context worth keeping. Advance the turns in the figure and watch the blocks disappear behind you.

Then there's the effort parameter, which is where the two versions of this model differ in a way callers can actually feel. GLM-5.3 accepts three levels, `low`, `high` and `max`, defaulting to `max`. And it **cannot be told to stop thinking**. The old way of disabling reasoning is gone, the launch post says requests that try it will fail outright, and migrating requires an explicit code change. Given that the Coding Plan bills output at 24 times the input rate and reasoning tokens are output tokens, a model that reasons hard unless told otherwise, and that cannot be told to stop, is a commercial decision as much as a technical one.

GLM-5.2's contract is stranger and better as a teaching example. It advertises **seven** values and honors two. Ask for `none` or `minimal` and thinking is skipped. Ask for `low` or `medium` and you silently get `high`. Ask for `xhigh` and you silently get `max`. A caller who carefully tunes down to `low` to save money is billed for `high` and never told. The template corroborates it from the other side: it can only emit `High` or `Max`, which is why the seven-value enum has two real settings. Send a value in the figure and watch what the server does with it, against what the documentation promised.

I should flag how thin the ground is under all of this. The template is GLM-5.2's, read from its tokenizer config, because no GLM-5.3 repository exists to read one from.[^15] And Z.ai's own documentation contradicts itself across four pages: one says the effort parameter is supported only by GLM-5.2, another says 5.2 and above, a third still documents the disable flag that the launch post says now fails, and the release notes page does not mention GLM-5.3 at all. The reconciling fact is that the GLM-5.3 API has not shipped. The three-level contract is currently the *subscription plan's* contract, and the API reference has not been written yet.

## The release

Everything up to here has been about a machine that did not change. This chapter is GLM-5.3.

Z.ai is unusually direct about what it did, and I'd take the sentence at face value: **"It carries over the RL strategies introduced in GLM-5.2, including SAO with compaction."**[^1] No new algorithm. No new architecture. What scaled was the environments, and the blog states the thesis in a form that I think is the most interesting claim in the release:

> "As agent capability improves, much of the difficulty in scaling post-training moves from the model to the environment."

If that's right, it relocates the hard part of building frontier models. For a decade the bottleneck was the model: better architectures, better optimizers, more data. The claim here is that once a model can plan and use tools competently, progress is limited by your ability to manufacture *situations* worth learning from, at volume, with a grader that can be trusted. That's a simulation problem and a software engineering problem more than a machine learning one.

I want to be careful, because it's also a convenient thing for a lab to say when its architecture is frozen. But it's testable in principle, and the pipeline Z.ai describes is specific enough to argue with.

### The environment factory

Reinforcement learning needs three things: a task, a way to attempt it, and a grader. In the agentic setting the task is a job, the attempt is a long trajectory of tool calls, and the grader is the hard part. Get the grader wrong and the model optimizes the grader instead of the job, and that failure is as old as the field.

> **Interactive figure:** Candidate environments falling out at each gate: authored, judged solvable, verified without the answer key, shortcut-mined, and finally trusted.

Z.ai's pipeline has five stages, and I'll walk them in order because the ordering is the argument.

**A research agent authors the environment**, starting from patterns collected out of real work: multi-step dependencies, hidden state, things that cannot be solved by pattern-matching a single file. The stated ambition is tasks that "represent several days of work for an experienced engineer." Their example is machine-learning infrastructure work, where the model gets the same working environment a person would, with access to compute clusters, storage, internal documentation, codebases and past experiment results, and is asked to diagnose bottlenecks across a training stack, implement optimizations, run experiments, and deliver a measurable end-to-end speedup without breaking correctness. That is not a coding exercise. It's a week.

**A judge agent then attempts the task**, and this gate exists to catch a specific failure: environments that are broken rather than hard. An automatically authored task can be unsolvable because a dependency is missing, a service does not start, or the described goal contradicts the state of the world. Training on unsolvable tasks teaches nothing and burns compute at the scale where compute is the constraint.

**A verifier is synthesized without access to the reference solution.** This is the subtle one and it deserves its own beat. If you write the checker while looking at the answer, you write a checker that tests for *that answer*. The model then learns to reproduce a particular solution rather than to solve the problem, and every alternative approach, including better ones, scores zero. Withholding the solution forces the verifier to be written against the task's actual success condition. It's the same instinct as writing tests before implementation, applied to a machine that will attack the tests with far more creativity than a person would.

**The verifier is gated on three checks before it's trusted.** The known-good solution must pass. Doing nothing must fail. And an unsolved intermediate state must fail. Those three catch, in order, verifiers that are too strict, verifiers that pass everything, and verifiers that reward partial progress they shouldn't. Anything that survives produces a binary reward, and a binary reward that has been adversarially checked is worth more than a rich reward that has not.

**Solver trajectories are then mined for shortcuts.** Once real attempts exist, you read them for cases where the model got paid without doing the work, and you close those holes. Reward hacking is discovered empirically. All five stages run inside one training framework that puts the trainer and the rollout engine in a single dataflow, so an environment plugs in as data generation rather than as a change to the training loop.[^16]

Watch candidates fall out at each gate in the figure and the shape of it lands better than a list of stages does. The pipeline is a filter, and the interesting question about any filter is what fraction survives it. Z.ai does not publish that.

That omission is representative. There's no environment count, no compute figure, no token count, and one statement about duration: "Over the past month we kept scaling on this stack." GLM-5.2 shipped on 16 June and GLM-5.3 on 14 August, which is 59 days. Either the run occupied roughly the back half of that window or the phrase is loose, and it's the only public statement of how long the work that constitutes this entire release actually took. Z.ai does concede the honest limitation, which I credit: these pipelines "still require a meaningful amount of human-in-the-loop work," and making them autonomous is named as future work rather than claimed as done.

### One rollout at a time

The algorithm underneath repays a close look, because it inverts a design everyone in the field converged on, and the reason comes out of scheduling.

> **Interactive figure:** Two training timelines side by side: one full of bubbles waiting on stragglers, one continuously fed. Then the trust region, where one method zeroes a gradient on only one side of the boundary and the other drops it on both.

Here's the setup in plain terms. Reinforcement learning on a language model means: sample an attempt, score it, and adjust the weights so that tokens which led to a good score become more likely. The catch is that "good" needs a reference point. A score of 0.6 means nothing until you know whether 0.6 is better or worse than this model usually does on this prompt. That reference point is called the baseline, and the difference between the score and the baseline is the advantage, which is the signal the gradient actually follows.

GRPO, the method most open labs use, gets its baseline for free by sampling a **group** of attempts at the same prompt and using the group's mean. Elegant, no extra machinery, and it has one operational flaw: you cannot take a training step until every member of the group comes back. Agentic trajectories vary wildly in length, so the short ones finish and sit idle while one straggler grinds through a long tool-use chain. Z.ai's description is blunt: "large portions of the GPU cluster idle."

SAO's move is to set the group size to **one**. Every trajectory becomes trainable the moment it finishes, and the cluster never waits. Two consequences follow, and both are more interesting than the speedup.

**The value model comes back.** With no group there's no group mean, so the baseline has to be predicted rather than measured, which means training a second network to estimate expected return. The field spent years removing that component, because critics are finicky, expensive, and unstable at the start of training when they know nothing. SAO puts it back and then spends real engineering on making it work: two value updates per policy update, fine-tuning the value head with attention frozen, and pretraining the critic at scale so it doesn't start cold. A reintroduction, with the reasons it was removed addressed one at a time.

**And the update rule changes shape.** Once training is asynchronous, a trajectory finishing now may have been generated by a policy several updates old. Correcting for that properly means knowing the exact probability the *generating* policy assigned to each token, which the paper calls computationally prohibitive, so SAO uses the rollout engine's own recorded probabilities as a stand-in and computes an importance ratio from them. When that ratio strays too far from one, the sample is stale enough to be dangerous.

What you do about it is the part I find clever, and the usual one-line version of it is wrong, including the one I wrote first. PPO, the standard, **clips**, and its clip is deliberately one-sided. It compares the raw objective against a version with the ratio pinned to the edge of the trust region and keeps whichever is *worse*. So a token that has drifted in the direction that would flatter the update gets its gradient zeroed, and a token that has drifted the other way keeps its gradient in full. Half of the suspect samples still vote.

SAO **masks** instead. Any token whose ratio lands outside the region is dropped from the gradient computation, in both directions, whether or not it flatters the update. The trust region stops being a pessimistic bound on the objective and becomes a filter on the data. Sweep the ratio in the figure across the boundary and watch which of the two leaves anything behind, on which side.

The paper sells stability rather than speed, and the number is stark: standard GRPO "suffers from a performance collapse at approximately 160 training steps," while SAO trains stably for a thousand.[^17] On long-horizon tasks where a single trajectory can run for hours, the ability to keep training past step 160 is worth more than any throughput multiplier.

Three caveats limit how far any of this can be pushed, and they matter enough to state here instead of in a footnote. **No GLM model is trained anywhere in the SAO paper.** Every experiment uses a 30-billion-parameter Qwen backbone. The connection to GLM is a single sentence in the abstract with no supporting table, figure, or section. The variant Z.ai says GLM-5.3 uses, "SAO with compaction," appears nowhere in the paper. The paper also contains no efficiency numbers at all, so every RL efficiency figure in circulation traces back to the blog. Cite it for mechanism. Do not cite it as evidence at GLM scale.

### Grading the student's own words

One more training idea earns a section, partly because it's a beautiful piece of reasoning about distributions and partly because I have to be honest that its role in GLM-5.3 is unclear.

> **Interactive figure:** Two distributions with different support, and where the gradient lands under each of three regimes.

Distillation means training a small model to imitate a big one. The classical version collects the teacher's outputs and trains the student on that transcript. And it has a structural flaw that took the field a surprisingly long time to name: **the student is graded on text it would never have written.** The teacher's transcript lives in a region of possible outputs that the student, left to itself, essentially never visits. So the student gets very good at continuing sentences it will never start, and at inference time it wanders into its own territory, where it received no supervision at all. Errors compound from there, because every wrong step moves it further from the region it was taught.

On-policy distillation flips which distribution the training data comes from. **The student samples.** The teacher then scores the student's own tokens, and a penalty derived from the disagreement rides along with the reinforcement learning advantage. The gradient now lands on text the student actually produces, the only text that matters. The figure draws the two distributions and their overlap, and under the axis it draws a row of dots: the tokens each regime actually trains on. Drag the supports apart and watch them. Under the classical version they stay parked under the teacher's hump however far the student walks away; switch to on-policy and they follow the student, and the ones that land where the teacher has nothing left to say turn red.

The extension is multi-teacher: score against several models at once and let the student learn from whichever is most informative per token. It generalizes from top-k probabilities to the full vocabulary depending on how much bandwidth you're willing to spend.

The honesty note. This is documented as a **capability of the training framework**, not as a stated step in GLM-5.3's recipe. No distillation stage, and no supervised fine-tuning stage, is described anywhere for this release. I'm explaining it because it's part of the stack Z.ai built and because understanding it pays off, not because I can tell you it ran.

### Two implementations of one model

The last piece of the release is filed under systems engineering, and it's actually a correctness result. It's my favorite finding in this chapter.

> **Interactive figure:** The same weights, two implementations, and the gap between their answers closing by four orders of magnitude.

Training and inference are different workloads, so labs run different code for them. Z.ai trains under Megatron and generates rollouts under SGLang. Same weights, same mathematics on paper, two separate implementations tuned for opposite goals. And they do not produce identical numbers. Different kernel choices, different reduction orders, different precision along the way, and floating-point arithmetic is not associative, so the two stacks assign slightly different probabilities to the same token given the same context.

Ordinarily that's a rounding error nobody cares about. In reinforcement learning it is not, and here's why. On-policy RL assumes the probabilities you're correcting by are the probabilities that actually generated the text. Compute the token's probability under the training implementation, and it's a *different* number from the one the rollout engine used when it sampled. The correction is now wrong, in a direction nobody controls. **Your nominally on-policy algorithm has quietly become off-policy, and no error is raised.** Everything runs. The loss curve looks fine. The gradient is subtly aimed at the wrong thing.

Z.ai drove the disagreement between the two implementations down to **one part in ten million**, which they describe as a reduction of more than 99.99 percent against their previous setup. And they report that these systems-level alignment fixes improved end-to-end reinforcement learning throughput on long-horizon coding tasks by **more than 2.3 times**, which is the only RL efficiency figure that exists in any source for this release.

The general lesson is one I keep meeting in different clothes: at this scale, the boundary between a numerical detail and an algorithmic assumption stops existing. A bug that would be invisible in any other context becomes the difference between the algorithm you designed and a different algorithm with the same name.

## The evidence

Every number in the last chapter came from Z.ai. So does every number in the launch table. This chapter is about what happens when you go check the *other* columns, which nobody does, and which turns out to be where the story is.

I want to be upfront about the conclusion, because the forensics can read as prosecution and that isn't the finding. Z.ai disclosed its own evaluation settings in more detail than most labs bother with. What it left unsourced is everybody else's numbers. The result is a table in which the vendor's own row is the most verifiable thing in it, and I think that's a general property of launch tables rather than a fact about this company.

### The table, and where its numbers came from

Eight columns: GLM-5.3, GLM-5.2, Kimi K3, DeepSeek-V4 Pro, Qwen3.8-Max, Claude Opus 4.8, Claude Fable 5 with fallback, GPT-5.6 Sol. Seventeen rows. The movements first.

```figure:sheet?key=movements&placement=wide
label: What moved in 59 days, and what it moved against
head: Benchmark | GLM-5.2 | GLM-5.3 | Best in the table
align: left | right | right | left
row: Terminal-Bench 3.0 | 4.6 | **28.3** | Sol · 34.6
row: DeepSWE | 46.2 | 66.9 | Sol · 72.7
row: SWE-Marathon | 19.4 | 42.5 | Opus 4.8 · 48.8
row: FrontierSWE | 67.5 | 78.1 | Fable 5 · 88.2
row: CyberGym | 77.2 | **84.5** | **GLM-5.3**
row: ExploitBench | 24.4 | 54.4 | Fable 5 · 78
row: ExploitGym, 2h / 6h | 29 / 39 | 105 / 130 | Sol · 216 / 293
row: AutomationBench | 26.2 | **48.2** | **GLM-5.3**
row: GDPval-AA, Elo | 1508 | **1769** | **GLM-5.3**
row: Agents' Last Exam | 23.8 | 28.5 | Sol · 28.6
note: Every number here is Z.ai's, including the comparators. The right-hand column is the best value in Z.ai's own table, not on any leaderboard.
```

Those are large moves for 59 days on a frozen base, and where they are largest is informative. The exploitation rows roughly doubled and tripled, which is where vulnerability data was deliberately added. Terminal-Bench 3.0 went up six-fold, from a floor so low that almost any improvement looks dramatic. The rows that barely moved are the ones already crowded near their ceiling: CyberGym gained seven points because everybody in the column is between 77 and 85, and Agents' Last Exam gained under five because the whole column sits within five points of itself. Post-training bought the most where there was the most room and the most data.

GLM-5.3 is first in Z.ai's own table on exactly three rows: CyberGym, AutomationBench, and GDPval-AA. The launch framing is considerably stronger than the table, and one thing it should not be called is best-of-open-weights, because in Z.ai's own numbers Kimi K3 beats it on four rows and DeepSeek-V4 Pro on two.

One of those rows needs a note before you can read it. GDPval-AA is scored in Elo, the chess rating, where a gap of a few hundred points means one side wins most of the time. Its scale is anchored so that **human experts sit at 1000**. So GLM-5.3's 1769 is not a percentage of anything. It sits 769 points above the human anchor, and on a standard Elo scale a gap that size corresponds to winning the head-to-head roughly 99 times in 100. The anchor is real and I checked it. What the head-to-head consists of, Z.ai does not say, and where the number came from is a worse problem that comes later in this section.

> **Interactive figure:** The whole table, with every cell recolored by where its number came from. Click a relabeled one and the column header changes to the model that actually produced it.

One column header deserves unpacking before the provenance work, because the parenthetical on it isn't decoration. Fable 5 ships with safety classifiers that can decline a request outright, and a declined request comes back as a *successful* response carrying a refusal. Anthropic exposes a server-side setting that catches those and routes them to a different model instead. So a row labeled Fable 5 can contain answers Fable 5 did not write. Anthropic quantifies it in the one place it publishes both numbers: on Terminal-Bench 2.1, **20.9 percent of Fable 5's trials hit a safety refusal and fell back to Opus 4.8.** One trial in five. Z.ai carries the label and never says which model its own fallback pointed at, or how often it fired.

Now switch the figure into provenance mode, where each cell is colored by its source rather than its value.

The GLM-5.3 column is **vendor-run and disclosed**: settings named, harness named, effort level named. Every closed-frontier cell is **unattributed**. No footnote, no citation, no link, anywhere in the post. Z.ai's footnotes name only its own model, plus two competitors on a single row. So the comparison rests on numbers whose origin is not stated.

Some of those numbers can be traced anyway, and the trace goes somewhere strange.

Four cells in the cyber section reproduce Anthropic's own system card **to the digit**.[^18] Two of them are Anthropic's Opus 4.8 figures, sitting in the Opus 4.8 column, correctly. The other two sit under a header reading `Claude Fable 5` and carry the numbers Anthropic attributes to a **different model**.

```figure:sheet?key=relabelled
label: Four exact matches, none of them under the right name
head: Benchmark | Z.ai's "Fable 5" cell | Anthropic's Fable 5 | Anthropic's Mythos 5
align: left | right | right | right
row: ExploitBench | 78 | *not published* | **78**
row: CyberGym | 83.8 | *not published* | **83.8**
note: The two remaining matches sit in the Opus 4.8 column, where they belong. These two do not.
```

Two independent benchmarks, four exact matches, no misses. The plainest reading is that the cyber comparator column was transcribed out of Anthropic's system card, and the transcription took Mythos 5's scores and printed them under Fable 5's name. Z.ai's own prose corroborates it from the other side, because the prose *says Mythos* for numbers the table labels Fable. There is no Mythos column at all.

So it matters what Mythos 5 is. Anthropic describes it in one sentence: it "shares Claude Fable 5's capabilities without the safety classifiers."[^19] Same model, same pricing, guardrails removed, available by invitation to approved organizations through a program called Project Glasswing. It was suspended for eighteen days in June by a US government export-control directive and restored on 1 July to approved US organizations only.

And Anthropic publishes no Fable 5 cyber numbers at all, for a reason it states plainly: Fable 5's classifiers fire on those evaluations, so there is nothing to report. The Mythos figures exist because Anthropic ran them, in its words, "with all safeguards turned off," alongside an explicit written warning that the results "may not be directly comparable to public leaderboard entries produced under vendors' deployed conditions." Z.ai reprints the numbers and drops the sentence.

Click one of those cells in the figure and the header flips to Mythos 5 while the caveat slides in underneath. GLM-5.3's CyberGym first place, 84.5 against 83.8, becomes a 0.7-point lead over an unguarded configuration of a model the public cannot buy, measured by its own vendor with the protections switched off. The honest sentence isn't "GLM-5.3 leads on CyberGym." It's "GLM-5.3 comes within a point of what Anthropic's restricted model does unguarded," which is a more alarming claim rather than a less impressive one.

That conclusion stops at the cyber rows, and the discipline matters. On Terminal-Bench the same test fails: Z.ai's Opus cell does not match the system card, or the leaderboard, or anything else. That row really is Z.ai's own harness, and the near-agreement there is suggestive at most.

Two more states in the figure. **Contradicted**, where a cell disagrees with the benchmark's own leaderboard, and **no such entry**, where the model does not appear on that leaderboard in any configuration. Five of six comparator scores on ExploitBench fall into the second category. Two whole rows have **no leaderboard at all**: Terminal-Bench 3.0's repository redirects elsewhere, and CyberGym's leaderboard is a 404. The row carrying the open-source-state-of-the-art headline is unverifiable in principle rather than merely unverified.

One row gets a provenance state all to itself, and it's the GDPval-AA row I promised to come back to. Z.ai's footnote on it does not say Z.ai ran the evaluation. It says *"Models are evaluated by Artificial Analysis"*, crediting an independent organization by name. Artificial Analysis has published nothing: its page for GLM-5.3 is a 404, and the model appears nowhere in its GDPval results. So the most striking single number in the table, the one that reads as a model beating human experts by 769 Elo, is credited to a party that has not confirmed producing it. I had this wrong in an earlier pass, and the correction runs in Z.ai's favor: I first wrote that Z.ai had run someone else's harness and reported its own result, which is not what the footnote says. Z.ai says a third party did it. That absence turns up again later in this chapter, doing more damage.

Then the toggle I'd build this figure for even if it had nothing else. Z.ai's comparator set stops at Claude Opus 4.8. **Claude Opus 5 exists.** Add one column and two of the three first places disappear: GDPval-AA 1848.77 against GLM-5.3's 1769, AutomationBench 50.3 against 48.2. Watch the crowns fall off.

Two things belong here for fairness, and a figure that omitted them would be dishonest. The rounding drift in the GDPval-AA row runs **against** Z.ai in every single instance: three competitor scores are rounded up, by between two and five points, from their true values, and none of GLM-5.3's are. That's sloppiness, and sloppiness that costs you is evidence against manipulation rather than for it. And there's one row run by a genuine third party, FrontierSWE, evaluated by an outside firm. It's the only externally run row in the table, and it's a row Z.ai loses, 78.1 against 88.2. Publishing it was a choice.

### The number that isn't in the table

The launch does not lead with any of that. It leads with a coding improvement of about fifty percent, and that number comes from somewhere the table cannot reach.

Its source is **Z.ai Code Bench**, which is Z.ai's own benchmark, held private and run in-house. GLM-5.2 scores 23.4 and GLM-5.3 scores 34.5, which is a gain of 47 percent, rounded up in the headline. And it arrives at *fewer* output tokens, 75,000 per task against 96,000, so the claim is as much about efficiency as capability. At the level below, GLM-5.3 scores 31.4 at around 50,000 tokens, against Claude Opus 4.8's 29.5 at 120,000. Scoring higher on under half the token budget is a real result if the measurement is real.

> **Interactive figure:** Five results plotted against the tokens each one spent. Up and to the left is better, GLM-5.3 moves both ways at once, and the vertical axis has no name because Z.ai never gave it one.

Before the caveats, look at what the figure's vertical axis is called. It isn't called anything, and that is deliberate on my part for a reason I get to third.

Three things about that measurement, in ascending order of how much they bother me.

Z.ai names its own ceiling in the same breath. Claude Fable 5 reaches **39.5** on this benchmark at maximum effort, five points clear of GLM-5.3, and Z.ai prints that. A vendor publishing a competitor's higher score on the vendor's own private benchmark is not the usual move, and after a chapter of provenance forensics it should count for something.

The reason for keeping it private is a good reason and an uncheckable one, and those turn out to be the same sentence. Z.ai says a private benchmark "reduces the risk of contamination from public test sets and gives us a more faithful measure of real-world user experience." The next section shows what public-set contamination does to a benchmark, so this is not a pretext. It is also a claim that nobody can check, forever, by construction.

And the headline is quoted on an axis nobody states. Z.ai says Code Bench evaluates agents "along two dimensions: end-to-end task completion rate and fine-grained checklist accuracy." It never says which one the percentages are. Completion rate is all-or-nothing; checklist accuracy awards partial credit; a 34.5 means very different things under the two, and Opus 4.8 and Fable 5 are placed on the same unnamed axis. I could not resolve it from anything published, and it is the fairest question to put to the vendor in this entire notebook.

### When the benchmark resets

One pair of numbers in that table does more to explain modern evaluation than the rest of it combined. GLM-5.2 scores **81** on Terminal-Bench 2.1 and **4.6** on Terminal-Bench 3.0. The model is identical. Only the benchmark changed.

> **Interactive figure:** Seventeen real leaderboard entries as a dot strip, and Z.ai's entire row floating above all of them. Then the version flips and the ground vanishes.

Terminal-Bench 2.1 is 89 fixed tasks, mirrored publicly on Hugging Face, and scores had bunched up near the ceiling: the public leaderboard tops out at 83.8, and vendor-reported numbers run higher still. A frozen public task pool degrades in a specific way: it stops measuring capability and starts measuring exposure. The tasks leak into training data, directly or through the enormous secondary literature of blog posts and solutions, and every lab's score rises for reasons that have nothing to do with the models getting better at terminals.

Terminal-Bench 3.0 is continuously authored and rotating, written by a consortium of data companies plus Nicholas Carlini, and hosted by the Laude Institute. Scores fall through the floor because the tasks are new. A drop from 81 to 4.6 is not a regression. It's the removal of a subsidy.

There's a second finding in this figure that I did not expect and that reframes the first. Put Z.ai's Terminal-Bench 2.1 row next to the official leaderboard's seventeen entries. **Z.ai's entire row floats above the whole board.** Its *lowest* comparator, GLM-5.2 at 81, outranks fourteen of the seventeen real entries, and its highest exceeds every published figure from any source. Meanwhile GLM-5.2 has never been submitted to that leaderboard; the only Z.ai entry among the seventeen is GLM-5.1, at 58.7.[^20]

That isn't evidence of fabrication. It's evidence that the whole row is a different harness, run in-house, plotted against numbers nobody else can reproduce. Which is the subject of the next section.

One awkwardness complicates any simple reading. Two of the labs whose models appear as comparators in Z.ai's table also sponsor compute for the benchmark on whose 3.0 row Z.ai claims the open-source lead. Everyone in this story is entangled with everyone else.

### The harness is part of the model

An agentic benchmark does not measure a model. It measures a model wearing a harness: the tools it's given, how errors are surfaced, how many turns it gets, how its edits are applied. And the harness is not a small term.

> **Interactive figure:** The same model, the same tasks, and one tool swapped. The capability number moves further than most models are apart.

An independent study changed **only the edit tool** and moved pass rate by up to **64.6 points**.[^21] Format mismatch alone, where the model emits patches in a shape the harness cannot apply, produced a 46.2 percent patch failure rate for one GLM model. Swap the tool in the figure and watch a capability number move without the model changing by one weight.

Hold that beside the launch table. On every percentage-scored row in it, the distance from the worst model to the best is smaller than 64.6. Which means the harness term can exceed the model term, and every cross-model comparison run under different harnesses is comparing two things at once without saying which moved.

Nearly every GLM-5.3 evaluation was run inside Claude Code, a rival lab's coding harness, at a pinned version. I read that as a fairness move: it's the harness the comparators were plausibly designed against, and using it costs Z.ai any home-field advantage. Disclosing the version is more than most labs do.

Three other methodology decisions run the other way, and all three are in Z.ai's own footnotes. On one benchmark, Z.ai **removed the anti-cheat import checks** the benchmark ships with. On another, it **replaced pattern-matching verification with a language model inspector**, which is a defensible change and also a change that makes the grader softer in a way nobody can quantify. And on the cyber benchmark, wall-clock time budgets were **normalized by throughput**, so a faster model gets more real time inside the same nominal budget.

That third one has a problem underneath it that took me a while to see. The throughput figures are attributed to Artificial Analysis, an independent benchmarking organization. **Artificial Analysis does not list GLM-5.3.** I raised that earlier about the GDPval row; here it does more work. I scanned their full models page, 1.29 megabytes of it, and got zero hits; the newest GLM entries are 5.2.[^22] The figure they do publish for GLM-5.2 is 127 tokens per second, and the figure Z.ai cites for 5.3 is 115, so it isn't a carry-over either. A first-party number is wearing a third-party label, and it's the number that sets the competitors' time budgets: normalized at 40 tokens per second against GLM-5.3's 115, one open-weights competitor gets roughly **2.9 times less real API time**, on a benchmark where extending the budget from two hours to six is worth 36 percent more solves.

The general form is the thing to carry away. **Every GLM-5.3 number in circulation today is first-party**, including the ones that look sourced elsewhere. I checked Artificial Analysis, OpenRouter, Hugging Face, GitHub, and Z.ai's own release notes. None of them had heard of it.

### The exploitation ladder

The cyber results are the reason this release has the title it has, and the benchmark behind them is better designed than most, so it pays to know what it actually measures.

> **Interactive figure:** Sixteen rungs from crash to full chain, with each model's reach drawn against the seed budget it was given. Equalize the budget and the gap narrows.

ExploitBench, from two researchers at Carnegie Mellon, takes 41 real bugs in the V8 JavaScript engine and grades attempts on a **sixteen-rung ladder**.[^23] That structure is the insight. Finding a crash is not exploitation. Neither is controlling a register. Real exploitation is a chain: trigger the bug, shape the heap, gain a controlled read, escalate to a controlled write, defeat the mitigations, and arrive at execution. Each rung is strictly harder than the last, and a scalar pass-rate would collapse the entire progression into a single bit.

GLM-5.3 climbs from 24.4 to 54.4 on that ladder in 59 days, more than doubling. The number credited to the top comparator is 78. And Z.ai's own account of why this happened undercuts the word "emergent" in the launch title: vulnerability discovery data and environments were **deliberately introduced into the training mix**. What surprised them, they say, was how fast the capability kept developing as training scaled, and that the model began "to reason across multiple stages of exploitation, forming coherent plans for complete exploitation chains." Deliberate ingredient, unexpected slope.

The benchmark's README asks labs not to run reinforcement learning on it. Whether Z.ai complied is not stated either way, and I'm not going to assume either answer.

Two corrections to the comparison, one in each direction, and I'll show both.

The 78 is not what it looks like. On the leaderboard, that entry was produced with a **five-seed budget under an automated retry policy**, a configuration the site itself flags as inflated relative to its standard three-seed protocol. Z.ai ran three revisions with no retries. So the gap GLM-5.3 is chasing is overstated in the competitor's favor, which is a rare direction for a launch table to err in. The figure lets you equalize the budget and watch the distance shrink.

And a correction that runs against my own argument, which I'm keeping in because leaving it out would be the sin this chapter is about. On the other cyber benchmark I initially reported that Z.ai's figure for a competitor contradicted the official submission by 41 percent. It doesn't. I had computed the wrong metric. That benchmark scores under at least four definitions, which diverge enormously: capture a flag by any bug at all, capture it via the *intended* vulnerability with mitigations off, with mitigations on, or across all profiles. The published headline metric is the second one, Z.ai used that one, and the number I thought was missing was mine. The second time in this research that a discrepancy turned out to be me comparing incompatible metrics.

What survives the audit on this row: the throughput provenance problem, the effect of the normalization on the two open-weights competitors, and several comparator cells that appear on no leaderboard anywhere. What does not survive: my own headline finding about it.

## What it means

Two things in this release point outside it. One is a ledger of real bugs in real software. The other is a piece of information that isn't in any document, and its absence is the finding.

### The bugs that waited forty-five years

Z.ai runs a public disclosure ledger, in Chinese only, tracking what its models have found in real codebases. I read it first-party, and the numbers reframe the cyber results from a benchmark story into something with weight.

> **Interactive figure:** Forty-five years of software on one axis. Each mark is a real defect, placed at the year it was introduced, with a line running to the year a model found it.

**2,436 vulnerabilities recorded.** Of those, 1,097 are rated critical or high, with a further 1,286 rated medium. They span **269 open-source projects**. The impact span is **45 years**, and the earliest defect traced runs back to **1981**. The mean time a vulnerability sat undiscovered before a model found it is **26.6 years**.[^24]

Sit with that last number. The average defect in this ledger had been sitting in shipped code since the late 1990s. It survived the entire history of open-source code review, outlived several complete turnovers of the maintainer base, and was finally found by a machine reading faster than any of them could. The figure draws each one as a line from the year it entered the codebase to the year it was caught, and the lines are long.

Where the ledger came from matters, and it corrects a cynical reading I initially had. This is not the exhaust of public inference. Z.ai says it has been working with several security teams in China since GLM-5.2, running its models against real-world codebases, and the ledger records the affected project, the severity, a CVE where one exists, and how long the defect had been present. So it's the output of an ongoing private engagement, which explains both why it's large and why only **53 of the 2,436 are publicly disclosed**: undisclosed findings are undisclosed because disclosure belongs to the security teams and the affected projects, on their timeline, not Z.ai's. That's the responsible way to run a program like this, and it also means 98 percent of the ledger is a count of things nobody outside can inspect.

There's a subtler consequence that Z.ai does not address. The same blog post says vulnerability discovery data and environments were fed into post-training. So the pipeline that produced the ledger and the pipeline that produced the training data are **the same pipeline**. Z.ai never says whether the codebases behind those 2,436 findings were excluded from the training mix. On a benchmark that would be contamination; here there's no benchmark, only a count, so it isn't misconduct. But it does mean the ledger cannot be read as a measurement of generalization. It's a record of what the system found, not evidence about what it would find somewhere new. That's the sharpest question nobody has asked, and I'd want it answered before treating the number as a capability claim.

Three threads about safety converge here, and each one is checkable.

Z.ai is **withholding the weights on safety grounds**, deferring the open release about two weeks for evaluation and hardening. This is the first release in the 5.x line to separate the announcement from the weights, and the model whose headline is offensive-security capability is the first one its maker has held back.

The comparator setting the frontier on these benchmarks is **a model shipped without safety classifiers, measured with its remaining safeguards off**, sold to approved organizations only, briefly switched off by a government directive. That is the current state of measured offensive-security capability: the best public numbers come from configurations the public cannot obtain.

And the demand for an unguarded version is already visible. Within hours of the announcement, someone created a gated repository named for an abliterated GLM-5.3 built for offensive cyber work. **It is empty**, and it has to be, because no weights exist. It's evidence of intent and nothing more, and it should be read that way.

The counter-current is the part that complicates any tidy conclusion, and it comes from practitioners. The censorship complaint in the first-week discussion runs toward **Western models refusing defensive security work**: bug fixes on security-related code, monitoring tools, ordinary defensive engineering, declined. So the same capability is simultaneously too dangerous to release, only fully measurable with the guardrails off, already awaited by people who want the guardrails off, and under-served for the legitimate uses. Every position in that sentence is held sincerely by someone.

### The silicon nobody names

The GLM-5 technical report is 187 authors and a full infrastructure section. It names no training hardware.

> **Interactive figure:** Every element of the training recipe that the report does disclose, and the one field left blank, against seven chips named for something else.

No GPU type. No cluster size. No GPU-hour figure. The infrastructure section describes techniques and stays silent on what they ran on. A report at this scale would normally name the hardware, often with the total accelerator-hours beside it, because at this scale the number is a flex. Here it's a blank.

The report does name seven chip platforms, all domestic: Huawei Ascend, Moore Threads, Hygon, Cambricon, Kunlunxin, MetaX, and Enflame. Every one of them is named for **inference**. Deployment, not training.

This is the concrete refutation of a claim I have seen repeated confidently, that GLM-5 was trained on Ascend hardware. The report does not say so. The only place those chips appear is the serving path, and the training hardware is simply not disclosed. I can't tell you what they trained on, and neither can anyone else outside the company, which is the point of the figure: it draws a recipe with one field left out. An omission that specific, in a section that describes everything around it, is the one thing in this release that cannot be an oversight.

The surrounding context is not hard to reconstruct. Export controls make training hardware a politically loaded disclosure for a Chinese lab, in both directions: naming Western chips invites scrutiny of how they were obtained, and naming domestic chips invites scrutiny of whether they were sufficient. Silence is the only move that costs nothing. What the report *does* disclose is generous by comparison, 28.5 trillion training tokens from a 27-trillion-token corpus, a full optimizer story, and an honest account of an approach that failed. The one field left blank is the one with a government on either side of it.

## The read

I've spent this notebook on a machine that did not change, and I should close by saying what I actually think about the thing that did.

As I write this, no independent measurement of GLM-5.3's **capability** exists. I want to be careful with that sentence, because the stronger version of it is false and the stronger version is what I wrote first.

Independent measurements do exist. They are all economic. A third-party agent evaluation ran GLM-5.3, got billed, and recorded the rate it paid: $1.68 per million tokens in, $5.28 out. Somebody's endpoint telemetry shows a 97.9 percent cache hit rate across 58 million tokens. LMArena has the model staged in its database as `glm-5.3 (max)`, with a null rating, no votes, and a flag saying users cannot select it yet.[^26] The world has measured what GLM-5.3 costs and how it is plumbed. Nobody outside Z.ai has measured what it can do.

The baseline that *does* have outside numbers is GLM-5.2: an intelligence index of 53 against Fable 5's 62 at an eighth of the evaluation cost, and an LMArena rating of 1471 in text against 1585 in code, ranked 33rd and 9th on 27,000 and 8,000 votes. That split says something on its own. This family rates far higher at coding than at general conversation, which is what the product line is aimed at, and it's a more honest summary of the model than any row in the launch table.

The first-week impressions are mixed in an informative pattern. The complaints cluster on agentic reliability rather than raw capability: tool calling that fails and then repairs itself, models that wander off task and build things nobody asked for, a suspicion that the benchmark numbers are ahead of the experience. One heavy user running about a billion tokens a week gave the most useful review in the whole thread, which was "10/10 if it had vision." The bull case, from the same threads, is that it's "still shy of Sol and Fable, but only just by a hair."[^25] Both readings hold together if the gap is small and concentrated in reliability. My guess is that it is.

There's a structural incentive here, named plainly by people better placed than me to name it: for labs in this position, benchmark scores map to capital. That doesn't make any particular number false. It does mean the prior on a launch table should be set by who benefits, and this chapter has shown what happens when you check.

So here's my read, in three parts.

**The architecture is genuinely good, and it is optimized for a regime nobody has measured.** MLA plus DSA plus IndexShare compose into a twenty-fold reduction in per-token prefill cost at a million tokens, and the whole chain is real, reproducible, and confirmed against three independent implementations. But the crossover points where that story becomes true sit above 290,000 tokens, and the deepest public measurement of any GLM sparse-attention model is a quarter of a million tokens on a hobbyist's needle test. The headline feature of this model has never been evaluated at its headline setting, by anyone, ever. That's not a criticism of Z.ai specifically. It's a statement about how far the field's claims have outrun its instruments.

**The post-training thesis is the most interesting claim in the release and the least checkable.** If the bottleneck really has moved from the model to the environment, the competitive picture changes shape: the advantage goes to whoever can manufacture verifiable long-horizon work at volume, which is an engineering and data-acquisition problem rather than a research one. It also implies the frozen base model was the point of the design: you don't need a new architecture if what you're scaling is situations.

**And the verification asymmetry is the durable lesson.** A launch table is a rhetorical object. The load-bearing part is not the vendor's own row, which was disclosed here in more detail than most labs manage. It's the columns nobody audits, which turned out to contain another company's system card with a different model's name on the header. I did not expect the strongest finding in this research to be about Anthropic's numbers.

### The wager

This notebook was written in the two days after launch, on purpose, before the weights land. That window makes something possible that explainers usually cannot do: state predictions in public, with dates, and be around to be wrong.

> **Interactive figure:** Five dated claims, each with the exact observation that would refute it, against a clock you can run forward past their dates.

Each card in the figure has two faces. The front is the claim; turn it over and you get the single observation that would kill it, which is the only thing that makes a prediction worth publishing. The strip above the cards is a clock. Run it forward and each claim crosses its own date into a lane that says nothing stronger than *past its date, still unanswered*, because that is the only honest second state for a piece written in front of the thing it is predicting.

**The weights land around 28 August 2026.** Z.ai said roughly two weeks, pending safety evaluation. The prior releases in this line shipped weights on announcement day, so this is the first time the promise and the artifact have been separated, and a missed date would say something about which of the two the safety language describes.

**They will load under the existing model class with no new engine code.** This is the real test of the whole thesis, and it's binary. Fifteen independent implementations run the GLM-5.x line today, and every one of them tells the releases apart by config values alone. If "the same base model as GLM-5.2" is true, GLM-5.3 loads under the same class, with no pull request anywhere, and the serving stacks stay silent. **A new model type, or any architectural pull request, refutes it outright.** I think the claim holds, mostly because no engine has needed to prepare for anything.

**The released config will differ from GLM-5.2's in bookkeeping fields only.** Version strings, maybe a name. Anything touching layer counts, expert counts, the indexer pattern, or the rotary base falsifies the frozen-base story in a way no amount of prose can rescue.

**No long-context benchmark will ship with the weights.** Not RULER, not MRCR, not LongBench. Z.ai has now shipped two consecutive releases advertising a million-token window and has published no measurement at any length for either of them. I would very much like to be wrong about this one, and it's the prediction I'd bet on hardest.

**The license will be MIT, and this is the card I am least sure of.** Every prior GLM release used it. But no license has been stated for 5.3, precedent is not a fact, and this is the first model in the line its maker has held back on safety grounds. A use restriction on a model marketed for finding exploits would be defensible and would also be new for this family, and it would quietly change what "open weights" means here. Refuted by any license carrying a use-restriction clause, and settled only by the file itself.

The loop closes where it opened. A release whose entire content is post-training, on a machine whose every number I could check to the byte, announced with a table whose comparator columns came from somewhere else, evaluated on capabilities whose consequences run to bugs older than the people fixing them, and none of it downloadable. The most checkable thing about GLM-5.3 is the part that didn't change.

The weights answer the rest. Check back in two weeks.

**Glossary**

- **DSA**: DeepSeek Sparse Attention. Instead of attending over the whole context, each query attends over a keep-set of 2,048 tokens chosen for it by a cheap scoring pass called the lightning indexer. Core attention stops growing with context. The indexer does not, which is the whole story of this notebook's middle chapters.
- **IndexShare**: GLM-5.2's name for reusing one layer's sparse-attention selection in the layers above it. Twenty-one of 78 layers compute fresh indices; the other 57 inherit from the nearest full layer below. The paper that introduced the technique calls it IndexCache and never mentions GLM-5.2.
- **MLA**: Multi-head Latent Attention. Keys and values are compressed through a low-rank bottleneck before caching, so one 576-number row per token per layer serves all 64 heads. It survives positional encoding only because the head is split into a part that rotates and a part that does not.
- **MoE**: Mixture of Experts. The feed-forward layer is cut into many small networks and a router picks a few per token, which decouples total parameters from per-token compute. GLM takes 8 of 256, plus one that always runs, and those banks are roughly 97 percent of the model.
- **MTP**: Multi-token prediction. One extra stored layer that drafts the next token before the model has committed to it, so a serving stack can verify several guesses in a single pass. In GLM it is applied recurrently, so drafting deeper costs compute and no memory.
- **ReLU**: The rectified linear unit, which passes positive numbers through unchanged and turns negative ones into zero. In the indexer it is applied per head before the heads are combined, so one head's negative evidence cannot cancel another head's positive evidence.
- **RMSNorm**: Root-mean-square normalization. It rescales a vector by its own magnitude without subtracting a mean or adding a bias, which makes it cheaper than the classical LayerNorm. GLM-5.x uses it everywhere except one 128-dimensional vector inside the indexer.
- **RoPE**: Rotary Position Embedding. Pairs of dimensions are treated as points on a plane and rotated by an angle proportional to the token's position, so the positional part of a dot product between two tokens turns on their separation and not on where either one sits absolutely. GLM rotates 64 of each head's 256 dimensions and leaves 192 alone.
- **RULER**: A long-context benchmark that generates synthetic retrieval, tracing and aggregation tasks at a chosen context length, so the same task can be measured as the haystack grows. It is the instrument behind the only published quality curve for this attention mechanism.
- **speculative decoding**: Having something cheap guess the next few tokens, then running the real model once over the whole guess to check them in parallel. Verification costs about what generating one token costs, because the expensive part was reading the weights and you only did it once.
- **YaRN**: A method for extending a trained model's context window by rescaling its rotary frequencies after the fact, needing far less training at the longer length than starting over would. DeepSeek-V3.2 uses it at a factor of 40. GLM-5.2 uses none at all and simply raised its rotary base instead.

[^1]: Z.ai, ["GLM-5.3: Frontier Coding with Emergent Cyber Capabilities"](https://z.ai/blog/glm-5.3), 14 August 2026. The launch post, and the only page carrying GLM-5.3's serving contract. It renders nothing to a plain fetch, so its benchmark table, its evaluation footnotes and its parameter documentation were recovered from the compiled JavaScript bundle. Even the announcement date is an inference, from CDN image timestamps; it appears in plain text nowhere.

[^2]: The same post. That sentence is the entire release, and every claim in this notebook about GLM-5.3's machine rests on it being true.

[^3]: My own reconstruction, run against [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2). HTTP range-requests pull the JSON headers out of the safetensors shards without downloading the 1.5 TB behind them; summing the products of 59,585 tensor shapes gives 753,329,940,480, matching Hugging Face's reported total to the parameter. The script ships with the research for this notebook and prints its own agreement check.

[^4]: xLLM's GLM model loader. Its comment records that GLM-5.2 shares `model_type: glm_moe_dsa` with GLM-5.0 and GLM-5.1, and that the releases are told apart by config values alone.

[^5]: GLM-5 Team, ["GLM-5: from Vibe Coding to Agentic Engineering"](https://arxiv.org/abs/2602.15763), arXiv 2602.15763, 17 February 2026, 187 authors. Read first-party from the arXiv HTML. Every quotation from it in this notebook is verbatim, including the layer count that disagrees with the checkpoint.

[^6]: DeepSeek-AI's sparse attention report for DeepSeek-V3.2. Source of the indexer's scoring equation, the ReLU rationale quoted as "for throughput consideration", the two-stage distillation recipe, and the 943.7-billion-token sparse adaptation budget that reproduces from its published step counts.

[^7]: Yushi Bai et al., ["IndexCache: Accelerating Sparse Attention via Cross-Layer Index Reuse"](https://arxiv.org/abs/2603.12201), arXiv 2603.12201, 12 March 2026, from Z.ai and Tsinghua. The source of every ablation number in the layer-pattern section, including the failed method it reports on itself.

[^8]: `tonyd2wild/GLM-5.2-NVFP4-KV-4x-DGX-Spark-300kctx-42tok-s` on GitHub, results dated 21 July 2026. A reproducible four-node recipe, with verbatim retrieval of a needle at roughly 100K depth inside a 249,945-token prompt, and a companion claim of decode staying flat to 638,976 tokens.

[^9]: `modeling_glm_moe_dsa.py`, the reference implementation in Hugging Face `transformers`. Source for the router's exact execution order, the absorption path, the constructor that overwrites `head_dim`, and the verbatim to-do conceding that the reference cache stores expanded keys and values rather than the compressed latent.

[^10]: `config.json` for [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2), plus the equivalents for GLM-5, GLM-5.1, GLM-4.7 and GLM-4.6. Every lineage claim here was diffed field by field against those files.

[^11]: vLLM and SGLang, their GLM and multi-token-prediction model files. Both implement draft-step index reuse; vLLM's comment that multi-token-prediction layers "always build a full indexer" is why the checkpoint carries indexer tensors on 22 layers rather than 21.

[^12]: The same reconstruction script as note 3. Its FLOP model, composition breakdown and crossover solver produce every number in this chapter, and it asserts the derived figures back against the constants they came from, because two earlier rounds of this research quietly drifted apart from it.

[^13]: OpenRouter's public endpoints API for `z-ai/glm-5.2`, enumerated 15 August 2026: 33 endpoints with declared price, context window and quantization on each.

[^14]: Unsloth's quantization notes for GLM-5.2, which measure top-1 token agreement against the BF16 checkpoint for each build.

[^15]: `tokenizer_config.json` for [`zai-org/GLM-5.2`](https://huggingface.co/zai-org/GLM-5.2). There is no GLM-5.3 repository, so there is no GLM-5.3 template to read.

[^16]: [`THUDM/slime`](https://github.com/THUDM/slime), Apache-2.0. Megatron for training and SGLang for rollout inside one dataflow, which is what lets an environment plug in as data generation.

[^17]: Zhenyu Hou et al., ["Single-Rollout Asynchronous Optimization for Agentic Reinforcement Learning"](https://arxiv.org/abs/2607.07508), arXiv 2607.07508, 8 July 2026. Every experiment in it uses a Qwen3-30B-A3B backbone; the connection to GLM is one sentence in the abstract with nothing behind it.

[^18]: Anthropic's Claude Fable 5 and Claude Mythos 5 system card, 317 pages, read first-party. Source of the ExploitBench and CyberGym figures Z.ai reprints, of the "all safeguards turned off" caveat attached to them, and of the fact that ExploitGym appears in the document zero times.

[^19]: Anthropic, ["Introducing Claude Fable 5 and Claude Mythos 5"](https://platform.claude.com/docs/en/about-claude/models/introducing-claude-fable-5-and-claude-mythos-5). Some sentences on that page exist only inside the embedded JSON payload, so a text-only fetch returns nothing, which is how I convinced myself once that a real quotation was fabricated.

[^20]: The Terminal-Bench 2.1 leaderboard, all 17 entries enumerated, plus [tbench.ai/contributors/terminal-bench-3](https://www.tbench.ai/contributors/terminal-bench-3) for 3.0's authorship, data partners and compute sponsors. Terminal-Bench 3.0 has no leaderboard at all.

[^21]: ["The Harness Problem"](https://stencil.so/blog/the-harness-problem). Substituting only the edit tool moved pass rate by up to 64.6 points, and edit-format mismatch on its own produced a 46.2 percent patch failure rate for GLM-4.7.

[^22]: Artificial Analysis, `artificialanalysis.ai/models`, regex-scanned on 15 August 2026: 1.29 MB of HTML and zero occurrences of GLM-5.3. The per-model page for it is a 404.

[^23]: ExploitBench, from David Brumley and Seunghyun Lee at Carnegie Mellon University: 41 V8 bugs against a 16-rung capability ladder, and a README that asks labs, verbatim, not to perform reinforcement learning on it.

[^24]: Z.ai's Security Disclosure Ledger at `cvd.z.ai`, read 15 August 2026. The site serves Chinese only; the English path 404s and the language parameter returns the same DOM.

[^25]: The Hacker News launch discussion, `item?id=49294997`. Every first-week impression quoted in this section comes from that thread, and not one of them is a measurement.

[^26]: Re-checked roughly 36 hours after the announcement. The billing rate comes from a third-party agent evaluation that ran the model and logged what it was charged ([`tangle-network/agent-eval#614`](https://github.com/tangle-network/agent-eval/issues/614)); the cache hit rate from endpoint telemetry posted to the same Hacker News thread; the staged entry from LMArena's own model list, where `glm-5.3 (max)` carries a null rating and `userSelectable: false` while GLM-5.2 sits at rank 33 in text and rank 9 in code. This is also where I had to correct myself: "no number about GLM-5.3 exists that Z.ai did not produce" is false as worded, and only the narrower claim about capability survives.