Occlusion of real world objects using three.js

Question

Im using three.js inside an experimental augmented reality web browser. (The browser is called Argon. Essentially, Argon uses the Qualcomms Vuforia AR SDK to track images and objects in the phone’s camera. Argon sends tracking information to Javascript, where it uses transparent web pages with the .js tag to create 3D graphics on top of the phone’s tape.) My question is about three .js, however.

The data that Argon sends to the web page allows me to align the 3D camera with the physical camera of the phone and draw the 3D graphics in such a way that they seem to fit the real world. I would also like some of the things in the physical world to close 3D graphics (I have 3D models of physical objects because Ive set the scene or because they prepared objects like boxes that Vuforia tracked).

I am wondering if people have suggestions on the best way to do this occlusion using three.js. Thank you

Answer 1

EDIT: it looks like the next version for the three .js (R71) will have an easier way to do this, so if you can use the dev branch (or just wait) you can make it much easier. See This post: occlusion of a transparent object three.js

---

MY ORIGINAL ANSWER (without using new features in R71):

I think the best way to do this (to avoid extra work by creating new rendering passes, for example) is to change the WebGL renderer (src / renderers / WebGLRenderer.js) and add support for the new object type, maybe call them "occlusionObjects" .

If you look in the renderer, you will see two current lists of objects, opaqueObjects and transparentObjects. The renderer sorts the rendered objects in these two lists to display the opaque objects first, and then the transparent objects after them. What you need to do is keep all your new objects in the occlusionObjects list instead of the two. You will see that opaque and transparent objects are sorted based on their material properties. I think here you can add a property to the object that you want to be an occluder (maybe "myObject.occluder = true"), and just pull those objects.

Once you have three lists, see what the render () function does with these lists of objects. You will see a couple of places with rendering calls, for example:

renderObjects( opaqueObjects, camera, lights, fog, true, material ); 

Add something similar to this line to disable writing to color buffers, visualize occlusion objects only in the depth buffer, and then turn on the color buffer recording again before rendering the rest of the objects.

 context.colorMask( false, false, false, false); renderObjects( occluderObjects, camera, lights, fog, true, material ); context.colorMask(true, true, true, true); 

You need to do this in several places, but it should work.

Now you can simply mark any objects in your scene as "occluder = true" and they will only be displayed in the depth buffer, allowing the video to show and close any opaque or transparent objects created behind them.