Combining multiple stencils in GL

Question

I want to set up several stencils in OpenGL and then draw them in several combinations.

For example: I have two rectangular stencils:

Then I want to attract:

• anywhere
• left rectangle (blue + purple)
• right rectangle (purple + red)
• middle rectangle (purple)
• whole colored area (blue + purple + red)

I found that it is possible to declare several stencils in different bits of the stencil buffer, but I do not know how to do this. How to configure glStencilOp and glStencilFunc for this?

Or can I (should I) use glScissor for this?

Answer 1

Currently, I don’t know if it is possible to configure the stencil buffer to complete all of the above 5 steps without making any changes to the stencil buffer between them. It would be easy if glStencilOp provided a bitwise OR, but this is not the case, and only using an increment or decrement will you have to draw the rectangles several times.

But if regions are always rectangles, why not just use the scissor test? Thus, the first 3 steps (or actually 2 and 3) can be done simply by setting the rectangle area with glScissor and turning on the scissors test ( glEnable(GL_SCISSOR_TEST) ).

For the middle one (step 4), you either calculate the purple intersection rectangle yourself, or use the scissor test again, or use the stencil test:

 glEnable(GL_STENCIL_TEST); glStencilFunc(/*whatever*/); glStencilOp(GL_INCR, GL_INCR, GL_INCR); //increase the stencil value //draw both rectangles glStencilFunc(GL_EQUAL, 2, 0xFFFFFFFF); //only draw where both rectangles are glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP); //don't change the stencil buffer //draw things 

So, first we draw both rectangles and increase the value of the stencil wherever they are drawn. Then we draw our stuff wherever the stencil value is 2, which means both rectangles were drawn.

For step 5, you use the same stencil buffer, but with

 glStencilFunc(GL_LEQUAL, 1, 0xFFFFFFFF); 

for the second pass. This way, you draw something wherever the stencil buffer is at least 1, which means that at least one rectangle was drawn.

For more than two rectangles, this can get complicated, and you need to play a bit to find the most optimal way.