Stencil buffer behavior

[bvssvni]

Because of the design constraints in APIs with PSOs, the behavior of Stencil in the simplified draw state is limited to the set of PSOs in the backend.

Supporting all stencil settings on graphics hardware would require either lots of initialized PSOs or creation of new PSOs by need. Therefore, I picked a design for clipping, which is the most common usage and allows the smallest overhead in the backend.

Stencil buffer format

A typical stencil buffer stores a byte (u8) per pixel in the color buffer. Unlike the color buffer, the information in the stencil buffer is not visible on the screen. It can be thought of as a hidden buffer to support more advanced rendering. In 2D the most common usage of a stencil buffer is clipping.

Piston-Graphics assumes the stencil buffer uses u8 format.

Using the stencil buffer

Example: You want to draw a rectangle, but only the part that overlaps an ellipse. This is how you do it:

1. Render an ellipse with Stencil::Clip(1). Writes 1 to the stencil buffer where the ellipse would appear. The color buffer does not change.
2. Render a rectangle with Stencil::Inside(1). This will only render the pixels where the stencil buffer equals 1.

Design

Values 0 to 255 can be written to the stencil buffer. This means you can use up to 255 different clip shapes at once, but you can not combine them with logical AND.

For example, if you have a shape A and a shape B, you can render inside A OR B by using Stencil::Clip(1) for both shapes.

In the example above, if you wanted to render inside A AND B you have problem. In an API that allows you to increment the stencil value, you could use 1 for A, increment 2 with B which would result in 3 in the overlap A AND B. However, since there is no Stencil::Increment this operation can not be performed.

The reason A AND B is not currently supported is to reduce the number of PSOs or overhead in backends required for full functionality. A PSO must be initialized with the stencil operation.

One can think of it as separating a plane into regions, where each region is assigned a color. You can have 256 different colors. 0 is used as the background color. When rendering a shape, you can choose one color to draw inside or outside. The other colors are ignored.

In most cases, you need only 1 clip shape. For interaction between libraries, use 1 whenever possible.