IOS10 + SceneKit: rendering video using a custom shader

Question

How to render a video as a diffuse texture on a node in SceneKit when using a custom shader? It sounds like a simple thing, but:

• SceneKit elements do not support AVPlayerLayeras input texture ( https://stackoverflow.com/a/1654775/ ... )
• Alternatives:
  • using a SpriteKit scene with SKVideoNode, but:
    • Starting with iOS 10, rendering with openGL does not work as soon as the SpriteKit script is invoked ( https://stackoverflow.com/a/1654775/ ... )
    • I couldn’t figure out how to pass the uniform to the shader modifier when using Metal ( https://stackoverflow.com/a/1654775/ ... ). Everything worked fine with openGL (on iOS 9 when SpriteKit was working). There is surprisingly very little documentation about this.
    • It seems that SCNProgram fails as soon as the video participates in the SceneKit scene ( https://stackoverflow.com/a/1654775/ ... ). Everything works fine with the same setup without video.
  • converts the video output to CGImage, transferring it to CALayer and transferring this layer to the video SceneKit node (SpriteKit is not involved). Details here , but not a viable solution, as soon as the video is a little large because all the output is copied to the image, and it is expensive.

SceneKit, opengl/metal, ...

, , SpriteKit Metal, SceneView , , , ...

:

Answer 1

, , SceneKit: https://github.com/team-pie/DDDKit