Why does ASTC use ISE when almost nothing else does?

Lobsters Hottest News

Summary

This technical blog post explains why the ASTC texture compression format uses integer sequence encoding (ISE) instead of simpler prefix coding. It compares the efficiency of ISE to prefix codes for small integer ranges, showing that ISE's savings are modest but still worthwhile in certain contexts.

<p><a href="https://lobste.rs/s/ek75hg/why_does_astc_use_ise_when_almost_nothing">Comments</a></p>
Original Article
View Cached Full Text

Cached at: 06/01/26, 02:26 AM

# Why does ASTC use ISE when almost nothing else does? Source: [https://fgiesen.wordpress.com/2026/05/29/why-does-astc-use-ise-when-almost-nothing-else-does/](https://fgiesen.wordpress.com/2026/05/29/why-does-astc-use-ise-when-almost-nothing-else-does/) The[ASTC texture compression format](https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.html#ASTC)has its[“integer sequence encoding”](https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.html#astc-integer-sequence-encoding)to send small integers with a uniform probability distribution within their range\. When that value range is \[0,2k\-1\] for some integer k, this is straightforward: just send the values with k bits each\. But ISE ups the ante by supporting not just power\-of\-2 sizes ranges, but also allowing a single prime factor of either 3 or 5 in the size of the range\. So we can, for example, have some value x in the range \[0,95\], 96 total, where the low 5 bits are sent regularly, and then we have the “high trits”⌊x/32⌋\\lfloor x/32 \\rfloorthat are always in the range \[0,2\]\. When the extra prime factor is 3, these trits are arranged in groups of 5\. The number of possible combinations of 5 values in the range \[0,2\] is 35= 243 < 256, so we can encode our 5 values in 8 bits\. For 5 values in the range \[0,95\], that means we spend 5\*5 \+ 8 = 33 bits; had we just used regular binary instead, we would have probably rounded up to 7 bits per value, so our 5 values would take 5\*7 = 35 bits\. There we go, 2 bits saved over 5 values, about 2/5 = 0\.4 bits per symbol, so why even ask the question in the title? The actual answer is at least slightly more interesting: because of course in practice this method isn’t competing with just rounding up to the nearest integer bit multiple, we could also use a prefix code\. With 3 values, which in the following I’ll write as “a”, “b” and “c” to avoid confusion with actual bit strings, there is only one real choice of prefix code: one of the values gets sent with a 1\-bit code and the other two values each get a 2\-bit code\. Something like You can change which of the three codes gets the 1\-bit code length, but it’s always one of them\. Our assumption is that all three values are equally likely, so the expected cost of a value ends up being 1/3 \* \(1 \+ 2 \+ 2\) = 5/3 bits, or about 1\.67 bits per symbol\. The ISE encoding that sends 8 bits for every group of 5 values averages 8/5 = 1\.60 bits per symbol\. \(For reference, the theoretical minimum is log2\(3\) ≈ 1\.585 bits per symbol\.\) And suddenly the difference looks a lot less drastic\. Compared to a straightforward prefix code, the expected savings shrink from 0\.4 bits per symbol down to about 0\.07 bits per symbol, so we expect to save a single bit slightly less often than one in every 14 values\. ASTC blocks are not large\. They have 128 bits, total, and of the modes that use ISE at all, at least 17 of those bits are, effectively headers\. With at most 111 bits left for payload, the largest number of trit values we can even theoretically send in a block, assuming we stick with the minimum possible \[0,2\] value range works out to… 69 \(I swear I didn’t fudge this\)\. If you ever managed to cram that many values into a block \(not actually possible in the format for reasons that don’t matter here\), ISE’s expected savings over the bog\-standard prefix code work out to under 5 bits, and because we have to round to an integer size at block boundaries, that means the trits can only ever save a maximum of 4 bits over a really elementary prefix code, in the most extreme case\. In practice, most blocks don’t actually have that many values, and many of them will in fact not even make it over the threshold where the expected saving crosses 1 bit\! Now, I hopefully have you convinced that the question in the title is at least somewhat meaningful\. Now clearly, eventually, with enough data, the savings from something like ISE are definitely noticeable\. It’s just that ASTC blocks are quite short and can’t send that many values to begin with; what do we care about the long run if we can never get there? Sure, we might save 1\-3 bits over the block as a whole, but that’s in a range where we possibly could’ve used a slightly more compact encoding in the header fields, maybe gotten rid of some rarely\-used options, and gotten those extra bits that way, instead of dealing with the extra complexity that ISE brings\. \(Which is, on the whole, not that big a deal, other parts of ASTC are also rather complex, but it’s still a question worth asking\.\) And now that I’ve convinced you that this is not quite as trivial a question as it may seem, time to fess up: I’ve not been entirely honest with you in this post\. I haven’t said anything outright wrong \(at least, not intentionally\), but I’ve been using the word “expected” multiple times, and that is doing a lot of heavy lifting here\. It’s true, for uniformly distributed random input data, we*expect*the a ↦ 0, b ↦ 10, c ↦ 11 prefix code \(or one of its permutations\) to average 1\.67 bits per symbol\. But it depends on the data\! If all the trits we send happen to be a’s, we’ll actually only use 1\-bit codes and send 1 bit per symbol for that exact input sequence\. Conversely, if there’s not a single ‘a’ in the data we want to encode, we’ll actually end up using 2 bits per symbol\. With ISE, the size doesn’t depend on the data\. You need to send*n*trits, I can guarantee you, sight unseen, that ISE will encode those in⌈8n/5⌉\\lceil 8n/5 \\rceilbits, no matter what the values are\. And that is the real reason to use something like ISE in ASTCs context; it’s not so much about the superior packing density compared to what a simple prefix code could do, it’s about the predictability\. ASTC actually really leans on this, in that it infers some coding parameters from however many bits are left after the headers and all the other data in the block have been accounted for\. The predictability is essential\. So, now to answer the full question in the title: ASTC uses ISE because it works a block at a time, has very few bits to play with, and*really*benefits from the size of an ISE code stream being completely predictable without knowing the data\. It’s not that the expected cost of using a simpler prefix code would change things that much by itself, it’s that the lack of predictability would complicate encoders even further and end up requiring extra header bits to compensate for the variability that would add to the cost\. Conversely, the reason we don’t see more ISE\-like things elsewhere is that prefix codes are,*on average*, quite good and most applications are perfectly content with that\. The ones that don’t want to or can’t afford either variability or the small expected waste are likely to take the plunge into “full” arithmetic coding \(or related techniques\) instead\. ### Addendum: quints I didn’t go into the other \(quint\) encoding which needs a set of 5 values which I’ll call a through e\. 53= 125 < 128 so three quints fit into 7 bits with almost no waste, giving 7/3 ≈ 2\.33 bits per quint, vs\. the ideal log2\(5\) = 2\.32 bits\. The corresponding prefix code will assign something like with two 3\-bit codes and three 2\-bit codes, making the expected cost \(2\*3 \+ 3\*2\) / 5 = 12/5 = 2\.4 bits per symbol\. Once again, about 0\.07 wasted bits per symbol, so the rest of the discussion doesn’t materially change\. The maximum number of quints that can theoretically fit into the at most 111\-bit payload section of an ASTC block is⌊111⋅3/7⌋\\lfloor 111 \\cdot 3 / 7 \\rfloor= 47, making the expected bit savings from ISE on a maximum\-length list of values 47 \* 0\.07 bits = 3\.29 bits, which rounds down to 3 \(we can’t use the partial bit\)\.

Similar Articles

Re-balancing Deflate Compression Levels

Lobsters Hottest

Klaus Post discusses the process of rebalancing deflate compression levels in the Go compression library to make speed/compression trade-offs more linear and intuitive.

SQLite compressed text-history prototypes

Simon Willison's Blog

Simon Willison experiments with storing SQLite text revision histories as compressed JSON arrays, comparing whole-blob and chunked storage approaches. A simulation of 1,000 edits compressed 20.4 MB of revision text into 80.3 KB using Zstandard.