A big part of creating a believeable and compelling 3D environment for gaming lies in the creative and realistic use of color, light, and shadow. Because shadows are themselves 2D projections of 3D objects, the human visual system can use them to provide additional cues to aid in understanding a scene. The end result is a picture with far more feeling of "depth" than if shadows are ommitted. 

  
Figure 1. Adding "Depth" with Shading 

Notice how the shadows make the "floating" crate appear more in the foreground in the second image. Also notice how the natural shading of corners reinforces the separation of the floor and walls, and emphasizes the subtle angles of the diagonal walls. (You may need to click through the thumbnails and compare the full size 640x480 images to see these effects clearly.) 

The rest of this article will discuss techniques for getting from A to B without sacrificing rendering performance. 

Color Models and Modes 

One common model for quanitizing color is 24 bit RGB. In this color model, 8 bits are used to represent each of three primary colors (red, green, and blue) which are then mixed by summation to produce the final color (e.g. taupe or moss). This color model allows for the representation of 2²⁴ (more than 16 million) colors, which approaches the color resolution of mainstream media such as 35 mm photographic film, or professional video. 

A second important color model is HLS or Hue, Luminance, and Saturation. In this model, the Hue number represents the color on a scale of red - orange - yellow - green - cyan - blue - magenta - red. The Saturation represents the "purity" of the color in terms of how much white or grey has been added, and the Luminence represents how bright or dark the color is. 

Colors can be converted from RGB to HLS using the following function: 
HLS24 RGBtoHLS(RGB24& rgb) 
{ 
   HLS24 hls; 
 
   BYTE Max = max3(rgb.B, rgb.G, rgb.R); 
   BYTE Min = min3(rgb.B, rgb.G, rgb.R); 

   hls.L = (BYTE) (240.0 * (Max + Min) / 510.0); 

   if (Max == Min) { // achromatic -> r==g==b 
      hls.S = 0; 
      hls.H = 0; 
   } 
   else { 
      double delta = Max - Min; 
 
      hls.S = (hls.L <120) ? (BYTE)(240.0 * delta/(Max+Min)) : (BYTE)(240.0 * delta/(510.0 (Max+Min))); double h, r, g, b, mx, mn; r="(double)" (rgb.R) / 255.0; g="(double)" (rgb.G) / 255.0; b="(double)" (rgb.B) / 255.0; mx="(double)" (Max) / 255.0; mn="(double)" (Min) / 255.0; delta="mx" mn; if (r="=" mx) h="(g" b) / delta; else if (g="=" mx) h="2.0" + (b r) / delta; else if (b="=" mx) h="4.0" + (r g) /delta; h *="40.0;" if (h < 0.0) h +="240.0;" hls.H="(BYTE)" h; } return hls; } 
 
Figure 2. HLS Color Model in Action 

To represent an image in 24 bit RGB, at least three bytes are needed for each pixel of the image. This is rather expensive in terms of disk space for long term image storage and video memory space for image display, so different video modes have been created as a compromise between space and color fidelity. For gaming, we are mainly concerned with 16 bit (sometimes only 15 bit) color and 8 bit color. 

Video modes for 16 and 15 bit color use only two bytes per pixel, but are capable of representing significantly fewer colors than the full 24 bit color model allows. These two video modes compromise by using fewer bits per primary color (usually 5-6-5 RGB for 16 bit or 5-5-5 RGB for 15 bit, although some hardware uses other encodings). To convert a 24 bit RGB value to 16 bit color, one simply divides (i.e. arithmetic shifts right) each component by about 8 (i.e. 2 or 3 bits), and then merges the results into a 16 bit word. Although 16 bit video modes can only represent about 64000 colors, this is usually sufficient to convey a realistic image with only minor color banding artifacts. 

Video modes for 8 bit color use only one byte per pixel, but can only represent 256 colors at a time. This is such a small number of colors that most images can not be displayed accurately. Fortunately, many interesting images have concentrations of colors in certain small ranges with other color values being under-represented. This allows for optimizing image quality with a color palette. 

A color palette is a limited collection of 24 bit color values that will be used to color the pixels of an image. In 8 bit video modes, each pixel of an image must be colored with one of the 256 colors in the current palette. If a forest scene is to be rendered, then a palette consisting of mostly greens and browns may be devised to maximize the fidelity of the image. 

Converting a 24 bit RGB color into an 8 bit color is a matter of searching the current color palette for the closest available match. This is usually done using a distance minimization algorithm, where the distance between two 24 bit colors is defined as: 
   RGB24  c1, c2; 

   double r = (c1.r - c2.r)/256.0; 
   double g = (c1.g - c2.g)/256.0; 
   double b = (c1.b - c2.b)/256.0; 

   double d = sqrt(r*r + g*g + b*b); 
Sometimes, it is more important to match Hue than Saturation or Luminance. In that case a weighted distance function may be preferable: 
   HLS24  c1, c2; 

   double h = (c1.h - c2.h)/240.0 * 2.0; 
   double l = (c1.l - c2.l)/240.0 * 1.0; 
   double s = (c1.s - c2.s)/240.0 * 1.0; 

   double d = sqrt(h*h + l*l + s*s); 
When rendering through a color palette, indirection is used. The 8 bit value of each pixel no longer directly represents the RGB components of the desired color (as in 24, 16, or 15 bit modes), but instead represents an index into the current palette of available color values. 

Most games (including the Alpha engine) use an 8 bit video mode for rendering 3D scenes. The remainder of this article will concentrate on 8 bit color, although the techniques described can be used in any color mode. 

The Lighting Problem 

If the light level is increased from "normal", the Luminance of the input color is increased. When the light level reaches maximum saturation, the input color is mapped to almost pure white. 

Finally, the resulting color must be mapped to the appropriate color model. In an 8 bit video mode, this means that the nearest match for the desired output color must be selected from the color palette. 

The Alpha engine employs the following light model: 
   void LightColor(HLS24 clr_in, double light, HLS24& clr_out) 
   { 
      clr_out = clr_in; 

      clr_out.L = clr_in.L * light/16.0; 

      if (light <16) clr_out.S="clr_in.S" / (2.0 light/16.0); } 

Color Palette and Look Up Table 

 
Figure 3. Color Look Up Table 

The palette appears as the middle row of colors in the CLUT rectangle. It was designed to offer ranges of light-to-dark variations of several common base colors: steel blue, gold, grey, green, tan, olive, red, pure blue, yellow, and so forth. 

The color look up table is used to implement the light_function described above. For each input color (X) and incident light level (Y), the CLUT contains the index of the color in the palette which most closely matches the desired output of the light_function. These values are pre-calculated by a program, lighttable, which implements the light model computations defined in the previous section. 

Use of a CLUT reduces the previous light calculation to: 
   screen[j][i] = CLUT[ light_level ][ texture[v][u] ]; 
In the next section, we will examine how to compute the appropriate light_level for each pixel. 

Lighting Techniques 

Flat Shading 

Flat Run Shading 

Flat run shading looks acceptable, and is even perspective correct if you don't look up or down. This technique was a good compromise at the time, but doesn't really work in true 3D 6 DOF environments. 
Gourad Interpolated Shading 
This is (more or less) the technique used by Descent. In this method, correct light levels are stored with each vertex of the polygon, and simple linear interpolation is used to compute the light level of each pixel between the vertices. Descent uses a slightly simplified version in which the light level is only computed for each run of 8 or 16 pixels instead of for each pixel. 

Interpolated shading looks pretty good, but is not perspective correct. Its main advantage is that it is supported in hardware on most 3D accellerators, and that it is reasonably easy to compute, even in software. 

Light Mapping 

Light mapping generally gives the best looking results. It is perspective correct, and allows for accurate placement of shadows. Diffuse and spot light sources can be simulated because the processing is performed off-line. The major disadvantages of light mapping are the increased memory requirements, and the difficulty of dynamically modifying the light maps. One additional disadvantage is that most 3D accellerators do not provide direct support for light map based shading. 

Figure 4 shows a sample light map from the scene rendered in Figure 1. The light map shown is from the wall directly "behind" the floating crate in the scene. The two shadows on the light map are cast by the crate, from two different point light sources. 

  
Figure 4. Sample Light Map and Surface 

Rendering with Light Maps 

In fact, even if both rendering assumptions were true in a given circumstance, it still would not be a good idea to implement the inner loop of the renderer using the code above. This form of the code is very inefficient as it unnecessarily repeats operations from frame to frame. The code implements the following algorithm: 

Find the texel at index (u,v) in the texture array. 
Find the lumel at the same index in the light map array. 
Find the color at index (texel, lumel) in the look up table. 
Draw the color on the screen. 

Happily, this also solves the problems caused when the texture and light map are at different sizes and resolutions. 

Surface Cacheing 

A surface cache is just an array of pre-lit, pre-tiled textures that are properly aligned for fast rendering. In the Alpha engine, each surface cache entry is a 256x256 pixel array aligned on a 64 Kbyte boundary. (Note: it is possible to make better use of memory than this, with some complication of the texture mapping code. In practice, Windows NT's virtual memory manager seems to do an acceptable job, so this optimization has been considered low priority.) 

In addition to the surface arrays themselves, the surface cache contains index information needed to implement the least-recently-used (LRU) replacement scheme. Each polygon (or surface) in the world must have a unique id that can be used to locate its surface information in the surface cache. Alpha uses a 32 bit integer for this purpose. In addition, the surface cache must store height, width, and mip number information about each surface, as well as its age in the cache. 

When the rendering pipeline needs to draw a polygon on the screen, it asks the surface cache for a pointer to the polygon's texture array, given the polygon's unique id: 
   texbits = surfcache->Get(id, 
                            lightmap, texture->name, mipnumber, 
                            width, height); 
If the surface cache already contains an entry with the requested id, then a pointer to the entry is returned immediately. On the other hand, if the surface can not be found in the cache (e.g. if this is the first time the polygon has been rendered), then the cache must load the surface data into an available entry. If there are no empty slots in the cache, then the oldest entry in the cache is "evicted" and replaced by the new data. 

The next three arguments to the Get() function describe the data to be loaded if the entry is not already present. The cache finds the light map and the appropriate mip map of the desired texture in the world database. These structures contain both texture data and the width and height for the texture arrays. 

The loading process begins by finding the larger of the texture and light map dimensions, and using these for the surface dimensions. Then the texture is tiled into the surface array within the boundaries of the surface dimensions. Finally, the light map is applied to the surface data using the color look up table as described above. If the light map is of lower resolution than the texture map, then an intermediate light map is constructed by interpolating the sparse light map values. 

Using the surface of Figure 4 as an example, the cache finds mip map number 1 of the texture, and the appropriate light map for the surface: 

  
Figure 5. Texture and Light Map 

Since the light map is larger, the cache uses it for the surface dimensions which are 128x128. The light map compiler always creates light maps which have dimensions that are powers of two. The texture is tiled into the cache entry, and the light map is applied to it: 

 
Figure 6. Completed Surface Cache Entry 

One additional complication arises when mip mapped textures are used. If the surface being rendered is distant from the view point, a larger mip number is selected by the rendering pipeline. The mip mapped version of the texture is of smaller dimensions than the original and it will therefore use less memory in the cache. 

The problem is that the light maps are not also mip mapped. This means that the texture map and light map are no longer at equivalent resolutions. The cache deals with this by sampling the light map every n^th lumel to scale the light map down to the appropriate resolution. This can cause some visible artifacts when the sampling interacts with a dithered light map: 

  
Figure 7. MIP Mapped Surface and Detail Showing Sampling Artifacts 

In practice, this still tends to look better than the "sparkling" effects of not mip mapping at all. 

Light Map Calculation 

Most of the light map calculation is done once, by the level editor (or an associated program such as Quake's light) when the level is compiled. The basic process is quite simple: The level editor starts by creating a lightmap for each polygon. Each lightmap is a rectangular array large enough to paint the associated polygon. The level editor may trade off image quality for reduced memory needs by computing a light level every n^th pixel. 

For each lumel in the lightmap, the level editor calculates a light level using a Lambertian illumination equation for each visible light source. The Lambertian (or diffuse) illumination equation is: 

I = I_a + f_att I_p k_d (N • L) 

This equation basically says that the reflected light at any point on a polygon is equal to the (reflectected) ambient light in the room, plus the reflected directional light. The amount of directional light reflected decreases according to the obliqueness of the incident ray; the more "straight on" the light ray falls, the more light will be reflected by the surface. 

Furthermore, the amount of reflected light decreases according to the attenuation function as the distance between the light source and the reflecting surface increases. In reality, light is attenuated proporational to the square of the distance from the light source. However, using a true square-law attenuation is problematic in an 8 bit color model. Therefore it is common to use an attenuation function of the following form: 

f_att = 1 / ( ad_L² + bd_L + c ) 

In C++, this calculation looks like: 
   // for each light source in the PVS 
   while (src != 0) { 
      if (src->visible) { 
 
         // for each lumel in the lightmap 
         for (int y = 0; y x0, src->z0, -src->y0); 

               // compute L and distance: 
               Point  vl = lpoint - test; 
               double dl = vl.length(); 
               vl.normalize(); 
 
               // compute attentuation factor and N dot L 
               // note that fatt is linear 
               double fatt = 0.1 + 0.5*dl; 
               double gain = DotProduct(vl, p->plane.normal); 

               // check for ray-polygon intersection 
               if (!lightShadows || !blocked(test, lpoint, p, polys)) 
                  light += src->light * gain/fatt; 

               // clamp light to reasonable values 
               if (light > 31) light = 31; 
               if (light <0) light="0;" // update the lightmap for the current light source lbuf[y][x]="light;" } } } src="src-">next; 
   } 
This loop must be repeated for each point being illuminated in the lightmap for each polygon in the level. 

To correctly compute shadow boundaries, this step must check the visibility of each light source for each lumel by determining whether a light ray cast from the light source to the lumel is occluded by any polygon in the level. This last step requires a moderately expensive 3D ray-polygon intersection test for each lumel times each light source times each polygon. In the sample code above, this operation is handled by the blocked() function. This routine iterates through all the polygons in the level and tests for ray-polygon intersection in 3D. 

Since most polygons will not occlude the light ray, we can improve performance by using a series of progressively more expensive and accurate tests. The first test checks whether the ray-plane intersection point falls between the two endpoints of the light ray. If it does, the second test checks whether the light ray pierces the cubic bounding volume of the polygon. If this test also passes, the third test checks to see whether the ray-plane intersection point lies within the polygon boundaries. A major performance improvement comes from using the PVS to ignore both light sources and polygons that are known not to be visible from the polygon being illuminated. 

Figure 8 shows the effects of using a lower resolution light map with simple linear interpolation. This is approximately equivalent to the light mapping style used in Quake. 

  
Figure 8. Low Resolution Light Map and Surface 

Figure 9 shows the results up to this point in the lighting algorithm with high resolution lightmaps. Each lumel in each lightmap is computed by finding the nearest integer to the true light level (which is computed as a double precision value). This results in visible and distracting "banding" artifacts where the light level suddenly changes from one integral value to another. This technique only produces acceptable results when used with "noisy" textures (like the carpeted floor), which mask the banding. 

 
Figure 9. Simple Lighting without Error Diffusion 

The last two steps in the lighting algorithm greatly improve the image quality with only a minimal performance penalty at level compile time: The first step is to pass the lightmap array through a digital smoothing filter. This blurs the edges of the shadows resulting in a softer, more natural look. Finally, the lightmap is converted from double precision to integral values by using alternating Floyd-Steinberg error diffusion. This step "dithers" the boundaries between adjacent light levels to produce smooth gradations of illumination. 

The smoothing filter blurs the edges of the shadows in a cheap simulation of area light sources. Correctly computing soft edges with area light sources in the illumination algorithm above means extra computation on each lumel to determine if it is partly shaded by a polygon. For this to be feasible, we should probably compute shadow boundaries in object precision using Weiler-Atherton or a "shadow volume" approach. 

The smoothing filter is a simple 3x3 digital filter with the following matrix: 

1/16	2/16	1/16
2/16	4/16	2/16
1/16	2/16	1/16

This filter produces a two-pixel blur on the edge of each shadow. (See the shadows along the right hand side of Figure 9.) A 5x5 filter would produce even softer edges, but for the moment I am satisfied with the look of the 3x3 filter. I may experiment more with this in the future, but it seems that the "right" approach is full radiosity which is much more expensive. 

The key visual enhancement comes from using a simple error diffusion to dither the light level boundaries. When a double precision light level (e.g. 12.48) is quantized to an integral value (e.g. 12), a fractional light level is "lost" in the process. This fractional light level is the error between the true value and what can be represented in a limited number of quanta. Error diffusion algorithms work by saving this error value and spreading it to neighboring pixels, thereby minimizing the error over a larger region. 

The Alpha level builder uses an alternating direction Floyd-Steinberg error diffusion to dither the lightmaps. The Floyd-Steinberg algorithm spreads the error to adjacent pixels according to the following diffusion matrix: 

-	-	-
-	*	7/16
3/16	5/16	1/16

At any point in the process, the error term from quantizing the pixel marked * is multiplied by the matrix factors and added to the neighboring pixels that have not yet been converted (i.e. when processing top-to-bottom and left-to-right, the pixels to the right and on the next row). To reduce visual artifacts, consecutive rows are processed in alternating directions (first left-to-right, then right-to-left). When processing in the alternate direction, the diffusion matrix is simply reversed. 

Conclusion 

  
Figure 10. High Resolution Lighting with Error Diffusion 

The use of high resolution lighting with accurate shadow volumes and error diffusion combine to create high impact images that heighten the sense of place and realism. The net effect is to draw the player more deeply into the game world and to produce a more satisfying experience. 

Hosted by Irth Networks - http://www.irth.net