15.2 Antialiasing

Antialiasing is an algorithm that attempts to minimize sampling errors that occur when an edge of a primitive is displayed on a raster image. Visually, these errors cause the edge to be stair-cased or look jaggy. For scenes with moderate complexity and/or animation, these jaggies are the source of high-frequency noise, which is annoying and distracting to users.

Figure 15-1 Edge With and Without Antialiasing

In Figure 15-2 "Unweighted Area Sampling", antialiasing is achieved by weighting the intensity of the pixel in proportion to the area of the pixel covered by the edge. In signal-processing terms, this is called "unweighted area sampling".

Figure 15-2 Unweighted Area Sampling

High-end graphics machines typically use an antialiasing technique known as super-sampling, in which the pixel is divided into a grid of sub-pixels. A color is computed for each subpixel and the subpixels that are covered by a primitive are averaged to produce the final pixel color. In the case where more than one primitive covers a pixel, each primitive's color is weighted by the number of subpixels it covers. Also, depth (Z) can be found for each subpixel which allows antialiased interpenetrations between primitives. While super-sampling is straightforward and effective, it is also expensive in terms of memory and memory bandwidth.For a 4x4 subpixel grid, 16 color and Z values must be stored for each pixel. In addition, to achieve required fill rates, each of these values must be accessed every clock.

Because the Nintendo 64 machine has very severe cost and memory requirements, a new and novel technique for antialiasing that avoided (as much as possible) the storage requirements of super-sampling but yet provided satisfactory antialiasing was needed. This method relies heavily on the notion that different objects have different "aliasing" needs, and that the hardware can be simplified by requiring that different RenderModes are configured as appropriate for a particular object. As well, there are display-order restrictions for rendering certain types of objects. For example, transparent objects must be rendered after all the opaque objects. Finally, it was recognized that antialiasing of silhouettes could be done as a post process during video output. A data flow diagram of the analogizing algorithm is shown in Figure 15-3, below. Note that this method requires, in addition to the pixel color and Z value, three bits of coverage value and four bits of deltaZ per pixel, quite small when compared with super-sampling methods.

Figure 15-3 Antialiasing Data Flow

The antialiasing data flow shows the most general case for Z buffered and antialiased primitives. Other techniques are possible. For example, if the database is sorted and rendered in back to front order, non-z-buffered antialiasing can be used. All of the various types of antialiasing are discussed in detail in Section 15.7 "Blender Modes and Assumptions".

For each pixel, a subpixel mask is computed. This mask is a 4x4 grid of bits where the bit is one if the subpixel is covered by the primitive and zero if the subpixel is not covered. The mask is converted to a coverage value by adding all the bits of the mask together. Since we only have three bits of coverage, the sixteen subpixels must be dithered to eight. The coverage value is optionally combined with the pixel's alpha. This is useful for antialiasing edges created by a texture cut-out. In the blender, the pixel color and the last value stored for the pixel in memory are combined. The blender also combines the pixel coverage and memory coverage and does Z buffering. The blender typically performs operations such as antialiasing the internal edges of objects and transparency. The new pixel's color, coverage, and Z are stored in the frame buffer. The Video Interface (VI) reads the pixel color and coverage and antialiases the silhouettes of objects.

In the next section we discuss each hardware unit in the antialiasing data path in isolation, before considering how these units work together to render a complete image.