SceneKit - get scene rendering from SCNView as MTLTexture without using a separate SCNRenderer

Question

My SCNView uses Metal as a rendering API, and I would like to know if there is a way to capture scene rendering as MTLTexture without having to use a separate SCNRenderer ? Performance drops when I try to render a scene using SCNView and redraw the scene by frame MTLTexture using SCNRenderer (I try to capture the output of each frame).

SCNView gives me access to MTLDevice , MTLRenderCommandEncoder and MTLCommandQueue , which it uses, but not to the base MTLRenderPassDescriptor , which I will need to get MTLTexture (via renderPassDescriptor.colorAttachments[0].texture )

Some alternatives I tried tried using SCNView.snapshot() to get UIImage and convert it, but the performance was even worse.

Answer 1

** A warning. This may not be the right method for the App Store. But it works.

Step 1: replace the nextDrawable CAMetalLayer method with a new one using swizzling. Save the CAMetalDrawable for each render cycle.

 extension CAMetalLayer { public static func setupSwizzling() { struct Static { static var token: dispatch_once_t = 0 } dispatch_once(&Static.token) { let copiedOriginalSelector = #selector(CAMetalLayer.orginalNextDrawable) let originalSelector = #selector(CAMetalLayer.nextDrawable) let swizzledSelector = #selector(CAMetalLayer.newNextDrawable) let copiedOriginalMethod = class_getInstanceMethod(self, copiedOriginalSelector) let originalMethod = class_getInstanceMethod(self, originalSelector) let swizzledMethod = class_getInstanceMethod(self, swizzledSelector) let oldImp = method_getImplementation(originalMethod) method_setImplementation(copiedOriginalMethod, oldImp) method_exchangeImplementations(originalMethod, swizzledMethod) } } func newNextDrawable() -> CAMetalDrawable? { let drawable = orginalNextDrawable() // Save the drawable to any where you want AppManager.sharedInstance.currentSceneDrawable = drawable return drawable } func orginalNextDrawable() -> CAMetalDrawable? { // This is just a placeholder. Implementation will be replaced with nextDrawable. return nil } } 

Step 2: Configure swizzling in AppDelegate: didFinishLaunchingWithOptions

 func application(application: UIApplication, didFinishLaunchingWithOptions launchOptions: [NSObject: AnyObject]?) -> Bool { CAMetalLayer.setupSwizzling() return true } 

Step 3: Disable framebufferOnly for your SCNView CAMetalLayer (to call getBytes for MTLTexture)

 if let metalLayer = scnView.layer as? CAMetalLayer { metalLayer.framebufferOnly = false } 

Step 4: In your SCNView delegate (SCNSceneRendererDelegate), play with the texture

 func renderer(renderer: SCNSceneRenderer, didRenderScene scene: SCNScene, atTime time: NSTimeInterval) { if let texture = AppManager.sharedInstance.currentSceneDrawable?.texture where !texture.framebufferOnly { AppManager.sharedInstance.currentSceneDrawable = nil // Play with the texture } } 

Step 5 (optional): You may need to confirm that the one you select on the CAMetalLayer you receive is your goal. (If more than one CAMetalLayer at a time)

Answer 2

Updated for Swift 4:

Swift 4 does not support dispatch_once (), and @objc is added to the replacement functions. The swizzle setting has been updated here. This is tested well for me.

 extension CAMetalLayer { // Interface so user can grab this drawable at any time private struct nextDrawableExtPropertyData { static var _currentSceneDrawable : CAMetalDrawable? = nil } var currentSceneDrawable : CAMetalDrawable? { get { return nextDrawableExtPropertyData._currentSceneDrawable } } // The rest of this is just swizzling private static let doJustOnce : Any? = { print ("***** Doing the doJustOnce *****") CAMetalLayer.setupSwizzling() return nil }() public static func enableNextDrawableSwizzle() { _ = CAMetalLayer.doJustOnce } public static func setupSwizzling() { print ("***** Doing the setupSwizzling *****") let copiedOriginalSelector = #selector(CAMetalLayer.originalNextDrawable) let originalSelector = #selector(CAMetalLayer.nextDrawable) let swizzledSelector = #selector(CAMetalLayer.newNextDrawable) let copiedOriginalMethod = class_getInstanceMethod(self, copiedOriginalSelector) let originalMethod = class_getInstanceMethod(self, originalSelector) let swizzledMethod = class_getInstanceMethod(self, swizzledSelector) let oldImp = method_getImplementation(originalMethod!) method_setImplementation(copiedOriginalMethod!, oldImp) let newImp = method_getImplementation(swizzledMethod!) method_setImplementation(originalMethod!, newImp) } @objc func newNextDrawable() -> CAMetalDrawable? { // After swizzling, originalNextDrawable() actually calls the real nextDrawable() let drawable = originalNextDrawable() // Save the drawable nextDrawableExtPropertyData._currentSceneDrawable = drawable return drawable } @objc func originalNextDrawable() -> CAMetalDrawable? { // This is just a placeholder. Implementation will be replaced with nextDrawable. // ***** This will never be called ***** return nil } } 

In the AppDelegate app:

 func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool { // Override point for customization after application launch. // Swizzle CAMetalLayer.enableNextDrawableSwizzle() return true } 

Updated to add the currentSceneDrawable property to CAMetalLayer, so you can simply use layer.currentSceneDrawable to access it, instead of having the extension that it is externally.