GLM-5.3

Seven hundred fifty-three billion parameters, zero architectural changes, and not one capability number anyone else can check.

Z.ai announced GLM-5.3 on 14 August 2026, under the title "Frontier Coding with Emergent Cyber Capabilities."¹ The company's own documentation opens with the whole release in one sentence: GLM-5.3 "uses the same base model as GLM-5.2," and "all improvements come from post-training."² The architecture did not change. The context window did not change. Not a single field of the config changed, because there is no config: no weights have been released, no technical report exists, no license has been stated, and the launch page's huggingface link is the literal string "#". The weights are promised roughly two weeks out, pending what Z.ai calls safety evaluation and hardening.

Three weeks ago I read Kimi K3 end to end and called it the best-documented frontier model in existence. This notebook is that one's mirror image. GLM-5.3 is, as of this writing, the least checkable frontier release I have looked at: every capability number in circulation was produced by the vendor, several of the comparison numbers turn out to be another lab's measurements wearing the wrong label, and the model itself cannot be downloaded, bought through an API, or benchmarked by anyone outside the company.

So this notebook reads what can actually be read. The GLM-5.2 checkpoint is public, and if Z.ai's sentence is true, it is GLM-5.3's machine, byte for byte. I reconstructed its parameter budget from the raw safetensors headers and it reconciles against Hugging Face's count to the parameter.³ The launch blog renders nothing to a plain fetch, so I recovered its benchmark table and its serving contract from the compiled JavaScript bundle. Four papers, three independent inference engines, and a stack of leaderboards fill in the rest. Everything below is one of two kinds of claim. Most of it is checked against an artifact I can point at: a tensor header, a config field, a line of somebody's inference engine. The rest rests on Z.ai's word, or on my own reading of the evidence, and I flag those where they occur rather than collecting them into a caveats section at the end.

One more thing before the machine. This notebook was written in the two days after launch, deliberately, before the weights land. That window is what makes it possible to do something explainers usually cannot: state a falsifiable prediction in public, with a date on it, and be around to be wrong. It is at the end.

The release you cannot see

A word on who this is, because it comes back near the end of this notebook. Z.ai is a Beijing lab with deep Tsinghua roots: its papers carry Tsinghua faculty as co-authors, and the training framework behind this release ships from the university's data-mining group. GLM is its flagship line, and it ships an architecture roughly once per major version and then rides it. GLM-4.6 and GLM-4.7 have byte-identical config files, the strongest form of "nothing changed" that exists. GLM-5 and GLM-5.1 differ in exactly one field, transformers_version, a bookkeeping string. And GLM-5.3, on the vendor's own statement, changes nothing at all from GLM-5.2. Two of the four releases in the 5.x line are architecturally invisible, and the pattern repeats one version back.

The one release that did change things is GLM-5.2, in June. Its diff against 5.1 has three real moves in it: the declared context window grew from 202,752 tokens to 1,048,576, the rotary base rose from one million to eight million, and a family of new config keys appeared that cut the model's sparse-attention indexer from 78 copies to 21. Everything this notebook says about GLM-5.3's machine is really a statement about that June checkpoint, carried forward on one sentence of vendor prose. The figure draws the 5.3 column dashed for that reason. There is no zai-org/GLM-5.3 repository, so there is no config to read; the column is 5.2's values plus a claim.

The claim does have corroboration of an unusually credible kind: code written by people whose only motivation was making the thing run. No inference engine anywhere carries GLM-5.3 support, and none needs to. A comment in xLLM's model loader says it plainly: GLM-5.2 shares its model_type, glm_moe_dsa, with GLM-5.0 and 5.1, and the releases are told apart by config values alone.⁴ The entire 5.x line is one architecture discriminated by numbers in a JSON file. Fifteen independent implementations run it, and not one has a glm5 file.

What exists of GLM-5.3, then, as of this writing: a subscription tier. The model is reachable only through the GLM Coding Plan, locked to a list of approved coding tools. The API is "coming soon." Z.ai's own release-notes page had not acknowledged the launch a day later, and its price list still stops at GLM-5.2. The announcement date at the top of this notebook is itself an inference, from CDN image timestamps, because it appears in plain text nowhere.

This is the first GLM-5.x release to separate the announcement from the weights. GLM-5, 5.1 and 5.2 all landed on Hugging Face the day they were announced, on a cadence of 51, 74 and 59 days. The two-week weights delay is new, and the stated reason for it, safety hardening, is one of the threads this notebook follows to the end. A model whose headline is emergent cyber capability is the first one its own maker held back.

One layer, whole

Before taking anything apart, here is the entire machine, working. Everything later in this notebook is a drill-down into one stage of this figure, so give it thirty seconds and just watch a token go through.

A token enters as an id and leaves as a distribution over 154,880 vocabulary entries. In between sit 78 transformer layers plus one extra. Each layer reads a running vector of 6,144 numbers, the residual stream, which is the token's working state as it moves up the stack, and each layer writes its own contribution back into it. The path is strict pre-norm: normalize the stream, apply attention, add the result back, normalize again, apply a feed-forward block, add that back. Two RMSNorms, two residual additions, nothing else. GLM-4.5 wrapped its layers in extra post-norms for stability; at twice the size, GLM-5.x dropped them.

The attention stage is the elaborate one, and it has two unusual organs. The stream is compressed into a small cached summary, 576 numbers per token, from which all 64 attention heads later reconstruct what they need. That is MLA, and it is why this model's memory bill is a fiftieth of what it would naively be. In front of the attention sits an indexer: a cheap scoring pass that reads the whole context and picks the 2,048 tokens the expensive attention is allowed to look at. That is DSA. Only 21 of the 78 layers own an indexer; the other 57 borrow the most recent selection.

The feed-forward stage comes in two kinds. Layers 0 through 2 are ordinary dense blocks. Layers 3 through 77 are mixture-of-experts layers: 256 separate small networks, of which a router chooses 8 per token, plus one shared expert that runs on everything. Those banks are where this model keeps 97.49 percent of its parameters, almost all of it asleep for any given token.

After layer 77, a final norm and an output projection produce the distribution. And then there is layer 78, which is not part of the stack at all: a complete spare layer, attention and experts and all, whose job is to draft the next token early so that serving can check several guesses in one pass.

Step the figure through those stages once and the proportions are the surprise: two norms, a compressed cache, a cheap scoring pass, and then a bank of experts that is most of the machine by weight and almost none of it by design effort. The rest of this notebook works through that figure stage by stage, in roughly that order.

A few oddities of the shipped checkpoint belong here before the autopsies, because each one carries information. The model contains one classical LayerNorm and one bias tensor, and they are the same object: the little norm on the indexer's 128-dimensional key. Everything else in 753 billion parameters is bias-free RMSNorm. My best reading, offered as a reading, is that the indexer is the one place where a small numerical error changes a ranking rather than a value: its output feeds a top-2,048 cut, and a token nudged below the line silently vanishes from attention's world. The one component whose mistakes are discrete got the one norm with a mean-subtraction and a bias.

Two norms that look like a safety feature are not one. The 2,048- and 512-wide norms inside the attention block sit on MLA's compression bottlenecks, and they are easy to mistake for QK-norm. GLM-5.x has no QK-norm anywhere. Then there are the counts, which all have to agree, and agreement is the closest thing a checkpoint offers to an audit: 79 attention blocks, 78 of them in the stack; 3 dense plus 75 expert layers; 21 indexers plus a 22nd hiding in the draft layer; 76 expert-router bias vectors, one per expert layer plus that same draft layer. Every one of those is a term in the budget the next section closes to the byte.

Where the model lives

A word about proportion, out loud, because the shape of this notebook could otherwise mislead you. This chapter and the three after it are the architecture, and the architecture is the part of GLM-5.3 that did not change. It gets the most room anyway, for one reason: it is the part that can be known. The GLM-5.2 checkpoint is downloadable, so every claim about the machine can be pushed one layer deeper, checked against tensor shapes, recomputed from a script. The part that actually changed, the post-training, is one blog post with no weights, no paper and no independent number, and it gets the space that evidence supports. Length here is a map of what is knowable, not of what matters.

One convention for all four, so I can stop repeating it. Unless a sentence says otherwise, every number in them was read off the GLM-5.2 checkpoint, its config, or the source of an engine that runs it, and you can recheck any of them. There are maybe half a dozen exceptions, they are all interesting, and each one announces itself where it sits.

A census that closes to the byte

Hugging Face reports GLM-5.2 at 753,329,940,480 parameters. I rebuilt that number from scratch: HTTP range-requests pull the JSON headers out of the safetensors shards without downloading the 1.5 TB of weights behind them, the headers list 59,585 tensors with their exact shapes, and summing the products gives 753,329,940,480. An exact match, to the parameter, and it is what licenses every other number in this chapter.³ The reconstruction ships with this notebook as a runnable script, and the figure below rebuilds its bars from the same per-layer constants.

The stored view is almost comically lopsided. Routed experts are 97.49 percent of the checkpoint. Attention, the mechanism this whole model line is famous for, is 1.73 percent. The embeddings and output head together are a quarter of a percent, and the 22 indexers, the organ two chapters of this notebook revolve around, are 0.0274 percent: about 206 million parameters in a model of 753 billion.

Flip to the active view and the picture inverts. A token touches 40,298,947,584 parameters, 5.35 percent of the model. That set is every layer's attention, the 21 indexers, nine experts out of 257 in each of the 75 sparse layers, the three dense feed-forward blocks, and the output head. Within that set, attention is 31.94 percent. On disk it was 1.73 percent. Nearly a third of the parameters this model actually reads for a token belong to the mechanism that is a fiftieth of it at rest, an eighteen-fold inversion, and it is the single most useful fact for reading everything that follows: the part of this model that barely exists on disk is a large fraction of what has to be hauled out of memory to use it.

Two details of the count settle real confusions. First, the number spread. You will see this model quoted at 744B, 750B, 753.3B and "40B active," and all of them are one model under different counting conventions. The 40,298,947,584 active figure includes the output head, which is a dense 6,144-by-154,880 matrix that runs on every token and alone is 2.36 percent of the active budget. Subtract it and you get 39,347,364,864, which is exactly the "39.35B active" Z.ai advertises. The embedding matrix, same shape, costs nothing per token, because looking up a row costs a memory read and no arithmetic.

Second, a forensic gem. Hugging Face's dtype breakdown lists exactly 19,456 parameters stored in full FP32 precision, in a checkpoint that is otherwise all BF16. That number is 256 × 76. The only FP32 tensor in the model is a per-layer vector of 256 expert-routing biases, there are 75 expert layers in the stack, and 75 does not divide 19,456. The 76th copy is inside layer 78, the draft layer, which is how you can know, from a dtype histogram alone, that the draft layer carries a complete 256-expert block of its own. Checkpoints leak structure through their bookkeeping.

One conflict belongs on the record, because it says something about reports versus artifacts. The GLM-5 technical report states, in a sentence I am quoting verbatim, that the model "reduces its layer count to 80" and totals "744B" parameters.⁵ Every shipped config says 78 layers, the tensor index contains nothing beyond layer 78's draft module, and the measured total is 753.3 billion. The checkpoint wins. A technical report is testimony; a safetensors header is evidence.

The cache is the model

Attention has a memory problem that compounds with context. To generate token one million and one, a model must consult keys and values for the million tokens before it, and the naive ledger for this model is brutal: 78 layers × 64 heads × 512 numbers per head × 2 bytes is 5,111,808 bytes of cache per token of context. At the full window that is 4.88 tebibytes, for one conversation, before a single weight is loaded.

MLA is the fix, and GLM's version stores 576 numbers per token per layer. Not per head. The layer compresses each token's key-value information into a 512-dimensional latent plus one 64-dimensional rotary key shared by all 64 heads, and caches only that row. The heads' actual keys and values are reconstructed from the latent on demand, or, as the next section shows, never reconstructed at all. The arithmetic: 78 × 576 × 2 bytes is 89,856 bytes per token, plus 5,376 bytes of indexer keys on the 21 layers that keep one, for 95,232 bytes all-in. Against the naive 5,111,808 that is a 53.7× compression. A million tokens of context costs 93.0 GiB instead of 4.88 TiB. Push the figure's context slider to the right and the naive bar leaves the frame while the MLA bar is still a stripe; there is no scale on which you can draw both at a million tokens.

Two versions of that ratio circulate and both are right, so let me pin which is which. The 53.7× above is the honest serving number: all layers, indexer included. The 56.9× you will see quoted is one layer's MLA row alone, 32,768 naive numbers against 576. Use the first for anything about memory bills. The 93.0 GiB carries two caveats of its own: it assumes BF16 cache entries, which serving stacks halve to FP8 on newer hardware, and it is what production engines achieve, not what the reference transformers implementation does. That implementation caches the fully expanded keys and values, MHA-sized, with a to-do comment in the source admitting it. Measure GLM's memory with the reference code and you will conclude this section is wrong.

Why does GLM use MLA at all? The GLM-5 report is unusually frank about this: in their experiments, "MLA with a 576-dimension latent KV-cache cannot match the performance of GQA with 8 query groups."⁵ The compression was costing quality, and the fix was an optimizer change rather than an architectural one. Muon Split orthogonalizes each attention head's projection block independently instead of treating the fused matrix as one unit, and that closed the gap. That sentence explains more about GLM's attention than anything else Z.ai has published, including the model's oddly shaped heads: 256-dimensional values against 192-plus-64 query-key dimensions are the settlement of a quality deficit, not a clean derivation.

One config field deserves a warning label. num_key_value_heads: 64, equal to the attention head count, looks like it contradicts everything above. It describes the logical heads after reconstruction, not what is stored. The cache is the 576 row.

The rotation that breaks the trick

MLA's economy rests on an algebraic move, and the move has one enemy. This section is the hardest idea in the model and the most rewarding one to watch happen, because it is a fact about matrix multiplication that you can see.

The move goes like this. An attention score is a dot product between a query and a key. In MLA neither of those exists in memory; both get built on demand from small cached vectors, each by multiplying with a fixed matrix. The query is one matrix times the query token's compressed vector. The key is another matrix times the key token's 512-number cached row.

Now write the whole score out as a single expression and look at what sits where. The two cached vectors end up on the outside, one at each end. The two matrices end up in the middle, back to back. And that middle pair is the same for every token in the sequence, because a projection matrix is a weight: it stopped changing when training finished. So you can multiply those two matrices together once, at load time, and from then on attend directly against the cached row, never building a single per-head key at all. That fold is called absorption, and it's the entire reason a 576-number cache can serve 64 heads. Drag the figure's first stage and watch the two blocks fuse.

Now add position. Transformers need to know where tokens are, and this family uses RoPE: rotate each query and key by an angle proportional to its position before taking the dot product. A rotation is itself a matrix, so writing the score out now gives you six things in a row instead of four. Reading from the query end: the query's cached vector, its matrix, its rotation, then the key's rotation, the key's matrix, the key's cached vector.

The two rotations meet in the middle and combine into one rotation by the difference of the two positions. That collapse is the elegant part of RoPE, and it's why the mechanism encodes relative position without storing anything. But look where it leaves the survivor. The combined rotation is now wedged between the two projection matrices, in exactly the spot where absorption needed them touching, and it is a different rotation for every pair of positions in the sequence. The figure's second stage lets you drag the two positions: the would-be fused block changes on every drag. A matrix that changes per pair cannot be precomputed once. Absorption dies, and the cache saving dies with it. Rotation does not commute with low-rank fusion.

The fix, inherited from DeepSeek and confirmed line by line in GLM's modeling code, is surgical. Split each head's 256 query-key dimensions into two parts that never mix. The large part, 192 dimensions, carries no position at all and flows through the absorbable path, where the fusion replays untouched. The small part, 64 dimensions, carries all of the position, and it cheats: its key is produced directly from the raw hidden state, bypassing the compressed latent, rotated once, and cached already-rotated. Because that rotary key never involves the up-projection matrix, there is nothing for the rotation to break. The score becomes an absorbable term plus a small positional term, summed.

The implementation details are quietly elegant. The 64 rotary dimensions are deliberately excluded from the latent's normalization, since they are not part of the latent. The rotary key is created with a head dimension of one and then broadcast to all 64 heads as a stride-0 view, an .expand() that copies nothing: 64 numbers per token per layer, read 64 times. Per-head rotary keys would have cost 4,096. That one broadcast is the difference between MLA surviving position and not.

Deciding what to look at

Attention's other problem is arithmetic. Every token scoring every earlier token is a cost that grows with the square of the context, and at a million tokens the square is monstrous: run this model dense at its full window and 98.56 percent of the per-token compute is core attention. The 753 billion parameters become a rounding error against their own quadratic term. GLM's escape is a chain of three mechanisms, and the property that makes the chain worth following is that each fix promotes a new bottleneck for the next one to attack. This is the heart of the machine, and it is where the research for this notebook kept turning up things nobody has written down.

Attention that skips

DSA, DeepSeek Sparse Attention, is a bet that most of a long context does not matter for any given token. Instead of attending to everything, each layer attends to a keep-set of 2,048 tokens, chosen fresh for every query by a cheap scoring pass called the lightning indexer. At the full window, 2,048 of 1,048,576 is 0.1953 percent. The expensive machinery reads one token in five hundred, and the whole question becomes whether the cheap machinery picks the right five-hundredth.

The indexer's shapes are the mechanism, and one of them appears in no writeup I could find. Its query side has 32 heads of 128 dimensions, read from the same compressed latent the main attention uses. Its key side is the surprise: the key projection is a single 128-dimensional head, shared by all 32 query heads. That is why the indexer's cache costs 128 numbers per token instead of 4,096, and it is why the indexer is affordable at all. DeepSeek's paper confirms the intent in passing, and the tensor header confirms the shape directly.⁶

There is a second economy in it, and it matters later. The indexer runs in FP8, both operands quantized, where the attention it gates generally does not. DeepSeek says so with visible satisfaction: because the lightning indexer "has a small number of heads and can be implemented in FP8, its computational efficiency is remarkable." Hold onto that, because this notebook is about to spend a chapter counting the indexer's share of the arithmetic, and a count of operations quietly assumes every operation costs the same.

The scoring formula repays writing out, because two details in it are easy to get wrong. Each of the 32 heads dots its query against a token's shared key. Each head's score is passed through a ReLU, then the 32 rectified scores are combined by a weighted sum whose weights the token generates from its own hidden state. The ReLU comes before the sum, per head, per key: a head's negative evidence is discarded before it can cancel another head's positive evidence. DeepSeek states the reason with disarming honesty: ReLU was chosen "for throughput consideration." Speed, not accuracy. The figure computes this pipeline literally, and one honesty note applies to it: GLM's learned indexer weights are not public, so the score distribution you see is synthetic. The selection arithmetic, the head structure and every count are real.

Then the detail that reframes the whole mechanism. The kernel takes the top min(2048, context) scores, so when the context is shorter than 2,048 tokens, the indexer selects everything, and DSA is exactly full attention. Serving stacks go further and skip the indexer below that line. Drag the figure's context slider to the left and watch sparsity switch itself off. Nothing sparse happens to your short prompts, ever; the entire apparatus is dormant until the context is long enough for it to matter. Sparsity here is a regime the model enters, at a context length you can point to.

GLM's DSA differs from DeepSeek's in scale and in appetite. Same keep-set of 2,048, half the indexer heads, 32 against 64, and a declared context 6.4 times longer, which stretches the same keep-set over a far larger haystack: against its own declared window DeepSeek keeps 1.25 percent, GLM keeps 0.195 percent. And one thing DeepSeek concedes in its own report deserves the emphasis it rarely gets: DSA reduces core attention from quadratic to linear, but the indexer that enables it still scores every token against every token. The quadratic term did not die. It moved into the cheap pass, and that fact sets up everything in the next section.

Which layers get to decide

Run the numbers on DSA at a million tokens and the fix reveals its own successor. Core attention collapses by a factor of 512, and the indexer, the cheap pass, becomes 88.35 percent of the remaining per-token compute. The thing that picks what to look at now costs more than everything it was protecting combined. GLM-5.2's response is called IndexShare: stop running the indexer at every layer.

The config ships a literal 78-entry array assigning each layer full or shared. The full layers are 0, 1, 2, and then every fourth layer from 6 to 74: twenty-one in all. A full layer computes fresh indices over the whole context and attends over its selection. A shared layer has no indexer whatsoever, no weights for one in the checkpoint, and inherits the selection from the nearest full layer before it. The paper behind the technique implements this as a single buffer, overwritten at each full layer and read by the layers above it, which means the whole mechanism costs one buffer instead of 78, and one conditional branch.⁷ The figure shows the buffer being stamped and re-read as you walk the stack.

Two misreadings to close off immediately, because both are natural and both are wrong. Shared layers still run full sparse attention; only the scoring pass is skipped, never the attending. Inheritance also runs backward, to the most recent full layer, not forward or nearest-in-either-direction. Layer 0 is always full so there is always something to inherit.

There's an oddity in how that 78-entry array gets used, or rather does not. Only the reference implementation reads indexer_types at all. The two engines that actually serve this model in production ignore it completely and re-derive the identical pattern from two integers, a period and an offset. So the config carries an explicit, per-layer, human-readable description of which layers think for themselves, and the code path that matters never looks at it.

What one shared buffer buys turns out to be three different savings, and the figure draws all three bars. Compute: 21 indexer passes instead of 78 is 3.71× fewer, which blends to 2.82× fewer total attention-side FLOPs at a million tokens; Z.ai advertises 2.9×, and my reconstruction lands within three percent of that, so the claim checks. Weights: 57 layers carry no indexer tensors at all. And cache, the saving nobody advertises: without IndexShare every layer would bank its own 128-number index key per token, 19,968 bytes; with it, 5,376. The total cache falls from 109,824 to 95,232 bytes per token, a 13.3 percent cut in the quantity the section on decode will show is the actual bill.

One config key tells a small, sharp story about roads not taken. Both major serving engines, before applying the periodic every-fourth-layer rule, first check a field called index_topk_pattern, which can hold an arbitrary per-layer pattern. That is the slot where a searched, non-periodic pattern would go, and the search is not hypothetical: the technique's own paper ships one. GLM-5.2 sets the field to null. The machinery for shipping a smarter pattern exists in every engine that runs this model, and Z.ai left it empty. Whether that was laziness or confidence is the subject of the next two sections.

Put the two config fields side by side and they make a small joke about each other. One describes the layer pattern explicitly, in full, and no production engine reads it. The other is the one every engine checks first, and it's empty.

A naming note, since the literature will otherwise fork on you. Z.ai's marketing calls this IndexShare. The paper, from Z.ai and Tsinghua authors, calls it IndexCache, never uses the word IndexShare, and never mentions GLM-5.2 or 5.3.⁷ One mechanism, two names, linked only by Z.ai's own citation.

And one sentence from the GLM-5 report reads differently a release later. It calls DSA "lossless by construction," on the grounds that the indexer selects without discarding long-range dependencies, and concludes that it can therefore be applied "to all layers with no quality degradation."⁵ GLM-5.2 then took the indexer out of 57 layers of 78. The two claims are compatible if you are careful: losslessness is a property of the selection, and IndexShare is about reusing a selection rather than making one. Still, a lab that had just finished arguing for indexers everywhere deleted three quarters of them within four months, and the report gives no hint that was coming.

How the indexer learns what to keep

Nothing so far explains why a 32-head scoring pass should agree with 64 heads of real attention about what matters. The answer is the best mechanism in this whole architecture, and I have not seen it explained anywhere outside the primary papers: the indexer is never trained by the language-modeling loss. It is a distilled student of the attention it replaces.

Stage one is a dense warm-up. Freeze the entire model except the indexer and run attention dense, no selection. At every step, take the real attention scores, sum them across heads, and normalize each query's row into a probability distribution over the context: that distribution is the target. Train the indexer, alone, to produce a distribution as close to that one as possible, scored by how many bits you would waste encoding the teacher's answer using the student's guess. That measure is the KL divergence, and it is the whole loss. The model is the teacher, the indexer is the student, and the lesson is literally here is what I looked at, learn to predict it. GLM-5 ran this for 2.84 billion tokens; DeepSeek's equivalent used 2.10 billion. Two labs, nearly the same modest budget, teaching the same imitation.

Stage two turns the selection on and trains everything at once, and its critical mechanism is a single graph operation. The main model now trains normally on its language loss, attending only over keep-sets. The indexer keeps training on its imitation loss, now computed only over the selected tokens. And the indexer's input is detached from the computation graph: gradients from the language loss cannot flow into the indexer, and the indexer's loss touches nothing but itself. Two objectives, two disjoint gradient paths, one forward pass. The figure draws the detach as a physically cut wire, and lets you reconnect it to see the failure the design prevents, each loss dragging the other's parameters toward a different goal.

Here the two labs diverge spectacularly. DeepSeek's sparse adaptation ran 943.7 billion tokens, a figure that reproduces from its published step counts. GLM-5's ran 20 billion, forty-seven times less, with the report stating flatly that they "find that it is enough."⁵ Whether it is enough is a live question this notebook returns to, and one label matters for honesty: those training numbers are GLM-5's. Nothing published describes how GLM-5.2's sparse attention was trained, at what length, or on how many tokens. The shipped config describes the machine; the training of the machine at its current scale is undocumented.

Which layers matter, and why the answer evaporates

If 21 layers get to keep their indexers, which 21? The paper behind IndexShare ran that question to ground, and its answer is the most epistemically interesting result in this notebook, because it depends on when you ask.

The figure has a switch for the training regime, and that switch is the finding. Ask it training-free, dropping indexers from a finished model with no retraining, and the choice of layers dominates everything. On GLM-5 itself, at a 200K window, a searched quarter-pattern gives up 0.4 points of long-context average, 78.4 to 78.0. A naive uniform quarter-pattern gives up 5.7, and one benchmark, GraphWalks, collapses from 92.7 to 74.9. The paper's own summary sentence: which indexer layers are retained matters far more than how many. Early layers are the most sensitive, and the stated mechanism is quotable: "their perturbations traverse the longest propagation path." An error in layer 2's selection compounds through 76 more layers; an error in layer 70's has nowhere to go.

Then ask it training-aware, distilling the model to expect sharing while it trains, and the entire finding evaporates. Take the paper's 30B testbed. A retrained model that keeps every indexer scores 51.0. Drop half of them on a uniform pattern, retrain, and it scores 51.6, above the model it was supposed to be approximating. Drop three quarters uniformly, which is the shape GLM-5.2 ships, and it scores 50.6, four tenths below the baseline. And the searched pattern, worth seven points in the training-free world, lands below the all-indexer baseline too. Read that again: after retraining, the careful pattern loses to the dumb one, and throwing away half your indexers beats keeping them all. The paper's explanation is the mechanism worth keeping: retrained shared layers learn to adapt their attention to inherited indices, and the joint adaptation "eliminates the layer-specific sensitivity entirely." The model absorbs the constraint. Which layers matter is a real, measurable property of a frozen model, and it is not a property of the architecture at all.