Android - draw a YouTube video on SurfaceTexture

Question

I am trying to draw a WebView on a SurfaceTexture so that I can render it in OpenGL.

So far, I have successfully played youtube videos in WebView in a standard way:

... webView.getSettings().setJavaScriptEnabled(true); if (Build.VERSION.SDK_INT < 8) { webView.getSettings().setPluginsEnabled(true); } else { webView.getSettings().setPluginState(WebSettings.PluginState.ON); } webView.setWebChromeClient(new WebChromeClient() { }); webView.setWebViewClient(new WebViewClient()); ... 

hardware acceleration is also included:

  <application android:hardwareAccelerated="true" ... 

And I also successfully rendered WebView in OpenGL (excluding video playback) based on this tutorial: http://anuraagsridhar.wordpress.com/2013/03/13/rendering-an-android-webview-or-for-that-matter -any-android-view-directly-to-opengl /

Additional initialization of WebView (so it is always a "portrait" and fits into the texture):

  Point p = new Point(); activity.getWindow().getWindowManager().getDefaultDisplay().getSize(p); webView.setLayoutParams(new ViewGroup.LayoutParams(px, py)); surfaceTexture.setDefaultBufferSize(px, py); 

Change the onDraw method for WebView:

 @Override public void onDraw(Canvas c) { try { Point p = new Point(); activity.getWindow().getWindowManager().getDefaultDisplay().getSize(p); Canvas canvas = surfaceTexture.lockCanvas(new Rect(0,0,px,py)); canvas.save(); canvas.translate(-scrollL, -scrollT); super.onDraw(canvas); canvas.restore(); surfaceTexture.unlockCanvasAndPost(canvas); } catch (Surface.OutOfResourcesException e) { throw new RuntimeException(e); } } 

When you try to play a YouTube video on a web view made using the above hack, the rectangle of the video is black (but you can hear the sound). I suspect that hardware acceleration does not work with surface texture.

So the question is: Is there a way to play YouTube videos using OpenGL texture?

Answer 1

The problem is this: the video accesses a separate window, which is located behind the view hierarchy window, and is not available for painting on your SurfaceTexture . To complicate this issue, WebView will position and scale the video window, not paying attention to the fact that it will be drawn on Canvas , different from the one that will be displayed on the screen. In addition, the video is displayed in native code, and you would not be able to draw before Canvas fast enough to support anyway (that the whole idea was to provide the video with a separate window).

Any satisfactory solution will almost certainly require the use of private APIs to access the video window in native code. Depending on how you plan to use SurfaceTexture , you may find a workaround that involves intercepting URLs in WebViewClient , removing any video tags before rendering, and placing videos in SurfaceView or TextureView under your control. You can then place it where it should be in relation to where you painted the SurfaceTexture , but this limits your use of the SurfaceTexture for the SurfaceTexture . However, this type of hack will be very ugly.

Simply put, I don't see a clean way to do this. This is a good idea, but it just doesn't work with the public API.

Answer 2

Try using lockHardwareCanvas added at API level 23.