C64 Demo Effects Explained: Rodents In The Attic

TL;DR: A Commodore 64 demo called *Rodents in the Attic* uses classic border-opening tricks, a VIC chip glitch called the ghost byte, clever sprite positioning, and tight raster-time coding to put colorful high-resolution lemmings in the border area. ## The Demo and Its Impossible Effect I made a demo for the Commodore 64 called *Rodents in the Attic*. In this video, I explain how it works. I put a link in the video description to the demo itself, and I highly recommend watching it first, because it has a cute story, and there will be spoilers in the explanation. There is also a bit of a magic trick in the demo: a seemingly impossible effect where lots of colorful high-resolution objects are moving around in the border area. I'll explain why that shouldn't work and how I was able to pull it off anyway. But the part that was most difficult to implement is actually over here, and I'll explain how that works, too. ## Quick Refresher: How the C64 Video Output Works The C64 has a main area of 320 by 200 pixels where you can display a picture or text. You can place up to eight so-called sprites, either in front of everything or behind the foreground pixels — in this case, behind the text. A sprite is 24 by 21 pixels, but you can make the pixels twice as big in the X direction, the Y direction, or both. The sprites you see in the demo are in high-resolution mode, where each pixel is either transparent or one particular color. There is also a multicolor mode with two bits per pixel, allowing three colors plus transparency at half the horizontal resolution. In front of everything is a border. You can select one of four different size configurations for the border and change its color, but you can't turn it off completely. Except you can, and I'll get back to that in a second. The border is there to help game developers: for instance, enemy sprites can appear from the edge of the display. ## PETSCII Graphics In the demo, the background graphics — the interior of the room as well as the characters — fill the entire main area. This is so-called PETSCII graphics, meaning it uses text mode and the built-in font from the C64 ROM. That font contains a selection of graphical characters, and it's an established art form to make pictures from this limited character set. The screen has room for 40 by 25 characters, and each character has an independent foreground color, but all characters must share the same background color. For this picture, I used medium gray as the global color, but the most common choice is black, which is also what I settled on this time. Using PETSCII is a stylistic choice, but one technical benefit is that it takes very little CPU power to animate these graphics, because you only need to update character references, not individual pixels. You don't need to count the characters to see that this picture uses the full main graphics area, because the basic home screen does, and its boundaries remain intact when the demo begins. But that means the lemmings are treading around in the border area, which is normally impossible — unless the border is turned off, or "opened," as we say in C64 lingo. ## Opening the Top and Bottom Border As the video signal is generated, one row at a time, left to right, top to bottom, the video chip inside the C64 — the VIC chip — keeps track of two internal memory bits, two flags. - One flag represents whether it's currently drawing the border or not. Let's call it B for border. - The other tracks whether it's currently in these parts of the screen where the border fills entire raster lines. Let's call that one the V flag, V for vertical. Whenever the B flag is low, the VIC chip displays whatever pixel is generated by the rest of the chip, which could be coming from a sprite or the main graphics area. But when the flag is high, the current border color is emitted instead. The following logic is hardwired into the chip: - When we reach the top edge of the visible area, lower the V flag. - When we reach the bottom edge, raise the V flag. These events occur once per video frame, so 50 times per second in Europe. The B flag sees more action: - Every time we reach the left edge of the visible area, if the V flag is low, lower the B flag. - When we reach the right edge, always raise the B flag, turning on the border. Because we can select among four different border sizes, and we can change this configuration while the video signal is being generated, we can manipulate these flags. To open the top and bottom border, we want to keep the V flag low at all times. So we prevent the VIC chip from ever raising it, which it does at the bottom edge of the visible area. We start with a tall border configuration and allow the VIC chip to generate video up to some point in a narrow band of the screen, then switch to a different border size so that the VIC chip never finds itself exactly at the bottom edge of the visible area, and never raises the V flag. Once we are safely past the critical band, we select a tall border again, so we can repeat the process. We have to jump through these hoops on every single video frame, but it's not very complicated, and we can do it from a raster interrupt handler. ## Opening the Side Border Opening the side border is more tricky, but the general principle is the same. Here, we want to avoid raising the B flag, which the VIC chip does every time it hits the right edge of the visible area. So we start with a wide border, and when the VIC chip is here, we flip to a narrow border configuration. The VIC chip's border logic never finds itself at the position where it should raise the flag, so it never does. Then we restore the wide configuration afterwards. Opening the side border is tricky for two reasons. First, we have to do it on every single raster line that we wish to open. Instead of 50 times per second, we have to execute this procedure over 15,000 times per second. That has a big performance impact and leaves less time for the CPU to do other things. Second, the timing is extremely critical. This border size reconfiguration has to occur on one specific clock cycle: cycle number 56 of every raster line. So we have to synchronize the CPU exactly with the operations of the video chip and count the clock cycles of our code. These techniques were discovered already in the 1980s. First, how to open the top and bottom border, and later how to also open the side borders. ## The Lemmings: The Heart of the Trick Even with the borders fully turned off, the main graphics area stays put in the middle of the screen. So the only thing you can display in the border area is sprite graphics. And sprites are subject to constraints: a maximum of eight sprites on any raster line, each no more than 24 pixels wide, possibly X expanded, but that makes the individual pixels wider. Multicolor mode makes the pixels wider still. So the lemmings in the top border can't be individual sprites, because: 1. There are too many of them. 2. They have more than one color each and are in high resolution. It seems to be impossible. There is one more feature of the Commodore 64 VIC chip that plays a part here: a hardware bug called the ghost byte. When you open the top and bottom borders, what you see in these areas isn't just a solid background color. Instead, you see an artifact of main-area graphics generation that is normally hidden by the border. Specifically, the last byte of memory that the VIC chip can address is displayed as a repeating pattern of eight black or transparent pixels. Typically, you'd set this so-called ghost byte to all zeros to see the background color, or all ones to get a solid black area. But you can set it to any pattern, and you can even change the pattern while the display is generated. You can't change the ghost byte on every single clock cycle — the CPU is too slow for that — but you can certainly change it once per raster line. Recall from the story that the walking lemming animation is eight pixels wide, a perfect fit for the eight repeating pixels of the ghost byte. This is all in glorious high resolution, but so far only monochrome graphics. You can put sprites behind the foreground text, and in the same way they can go behind the ghost byte. To cover the full width of the main graphics area, you need seven X-expanded sprites. But the lemmings have three colors — blue, green, and gray — so we need to put the sprites in multicolor mode. Now we have a problem, because the lemmings are full of fine high-resolution details. But because we use expanded multicolor sprites, we have to work with bulky four-pixel-wide chunks. This brings us to the heart of the trick, the key insight that made the whole demo possible. When I studied Gary Timmons' original lemmings walk cycle, as one does, I discovered that in each frame of the animation, nearly every row of pixels contains only two colors besides black, with a single boundary separating those two colors. For instance, one line contains a gray part and a blue part, and the separating boundary is here. Another line is green and gray, and the boundary is here. We can adjust the horizontal positions of the sprites from one raster line to the next. So if our blocky expanded sprites look like this, but we adjust the sprite positions on every line, we end up with this. Then we can apply the black ghost byte pattern to hide the parts that are outside the lemmings. The position adjustments are different for each animation frame. This is what it looks like without the ghost byte mask, and now with the mask. Going back to the source material, nearly all of the lines have two colors separated by a single boundary, but there are a few exceptions. For instance, this line: here I had to deviate from the original pixel data and make the tip of the foot blue instead of gray. The Commodore 64 produces an analog video signal where color information is bandlimited. So arguably, an individual high-resolution pixel doesn't really have color, only brightness. But another way to think of it is that I'm cheating. ## Hiding Individual Lemmings In the demo, you don't see all the lemmings all the time. There are gaps in the stream. The global background color is black, so transparent pixels in the multicolor sprites are indistinguishable from the black ghost byte pixels, and that's how I can hide individual lemmings. In fact, a large proportion of the available CPU power is spent on scrolling a mask of transparent pixels across the multicolor sprite graphics. ## The First Lemming: A Special Case The very first lemming in the demo walks right off the edge and falls down the side of the screen. This requires opening the side border, which requires CPU involvement on every raster line and generally hurts performance. But the top border raster code is already maxed out performance-wise. On every raster line, it needs to update the ghost byte as well as the X position of each and every multicolor sprite. So the timing is very tight, and there's no room for opening the side border as well. So that first lemming is an exception, displayed in an entirely different manner: using a gray, a green, and a blue high-resolution sprite on top of each other, in order to leave enough clock cycles for the CPU to open the side border. A hole is then dug to divert the stream of lemmings, so we don't have to open the side border in the top border while we're busy updating sprites and ghost bytes. Furthermore, the hole is dug early enough that we don't need to cover the full width of the main area graphics with sprites. It's sufficient to place six X-expanded multicolor sprites in the top border, and that leaves two sprites free to implement the falling lemmings. These are rendered using a black high-resolution sprite in front of an unexpanded multicolor sprite. You'll notice a rhythm where they alternate between falling into the line and out of it. ## The Real Challenge: Lower Area, Side Border, and Bad Lines That's finicky and all, but the real challenge is in this area, where we have to repeat the trick from the top border — albeit only for a short stretch — while simultaneously opening the side border and without any help from the ghost byte. Remember, the ghost byte is a glitch in graphics generation that happens outside the main graphics area. But here, things are working fine and there's no ghost byte. Instead, we put a multicolor sprite here, and then we use regular character graphics with black foreground color to emulate the ghost byte pattern. That means we no longer have to update the ghost byte on every raster line, but we still have to adjust the X position of the sprites and open the side border on every raster line. When the lemming walks outside the main area, we can no longer use character graphics for the black mask, nor the ghost byte. So here we need a second sprite in high-resolution mode to provide the black mask. Then we need two more sprites for the falling lemming, again alternating between falling into the line here and out of it in the side border. What complicates matters is this little thing called bad lines. Whenever a new row of characters begins — for instance, right here in the middle of our time-critical lemming drawing routine — the VIC chip suspends the CPU for 40 clock cycles in order to fetch new character data from memory. This makes the timing even more constrained than it already was. To make a long story short, we can only display a maximum of four sprites while simultaneously opening the side border on a bad line. We have four, so that's all right, but that means we're at the limit. Furthermore, on a bad line, there's no time to update the sprite X coordinate. But we are lucky: the third and fourth rows of pixels in the original animation data can use the same X position in all frames except one, and there we end up with just one wrong pixel. ## PETSCII Font Modification and Sprite Reuse There's another complication. These are PETSCII graphics drawn in text mode using the default ROM font. I actually make a copy of the font in RAM, so I can modify it. While the picture is made from graphical characters, and the text bubbles contain inverted letters of the alphabet, I never use the non-inverted letters of the alphabet. So I can repurpose those character codes in this part of the screen where the tunnel will go. Since the global background color is black, I can just clear the relevant pixels in the font definition as the second lemming digs and bashes its way through. But as soon as the bulk of the lemmings reach this area, I have to use the characters in these locations to emulate the ghost byte. That means they need to have a black foreground color. But the global background color is also black, so this whole area would end up black. How can I display the remaining parts of the background picture — the orange and blue pixels that didn't get bashed away? I can't change the foreground color in the middle of these character cells without introducing another bad line, and that would ruin the timing. So the remaining option is to use sprites, but I can't add more sprites in this area, remember? Instead, I reuse the multicolor expanded sprite and the black ghost-byte-emulating sprite to display the orange and blue parts, and then I reconfigure them just in time for the stream of lemmings. ## Dynamic Side Border Code Opening the side border is expensive in terms of clock cycles, so I don't want to open any more raster lines than I have to. But the lemmings in the side border are falling, so my code for opening the side border must be able to run at any vertical position. Meanwhile, the bad lines are stationary, and they affect the timing of the code. So the code for opening the side border in the lower part of the screen detects dynamically when a bad line is due and takes a differently timed code path. Sometimes it even needs to run here, overlapping with the opening of the vertical border, and that special case must be handled as well. The most complicated effect entanglement occurs right here, where the non-moving timed code we talked about earlier — which is already quite complicated — needs to pass control to the moving border-opening and bad-line-dodging code running here. For that, in the end, I had to resort to look-up tables and multiple code paths to handle each animation frame separately. ## The Sound Effect At the end of the show, when the final lemming explodes, you hear a distinct splat. That sound is generated by the SID chip, obviously, but it's not something you could produce from a traditional music play routine running once per video frame, because here you have to update the pitch registers very rapidly for the full duration of the sound effect. So at this point, the display is briefly turned off and only a white border color is shown. All sprites and bad lines are turned off, leaving every last clock cycle