Android TextureView / Graphics / Graphics

I am trying to make a paint / paint application using TextureView on Android. I want to support a paint surface up to 4096x4096 pixels in size, which seems reasonable for my minimum target device (which I use for testing), which is the Google Nexus 7 2013, which has a good quad-core processor and 2 GB of memory.

One of my requirements is that my view should be inside the view, which allows it to be scaled and taken out and tinted, which is all the code I wrote (I think UIScrollView from iOS).

I tried using a normal View (not TextureView ) with OnDraw, and the performance was absolutely terrible - less than 1 frame per second. This happened even if I called Invalidate(rect) with only the rectangle changed. I tried disabling hardware acceleration for the presentation, but nothing was displayed, I suppose, because 4096x4096 is too large for software.

Then I tried using TextureView , and the performance is a little better - around 5-10 frames per second (still awful, but better). The user draws a bitmap that is then pulled into the texture using a background stream. I use Xamarin, but hopefully the code makes sense for Java people.

 private void RunUpdateThread() { try { TimeSpan sleep = TimeSpan.FromSeconds(1.0f / 60.0f); while (true) { lock (dirtyRect) { if (dirtyRect.Width() > 0 && dirtyRect.Height() > 0) { Canvas c = LockCanvas(dirtyRect); if (c != null) { c.DrawBitmap(bitmap, dirtyRect, dirtyRect, bufferPaint); dirtyRect.Set(0, 0, 0, 0); UnlockCanvasAndPost(c); } } } Thread.Sleep(sleep); } } catch { } } 

If I change lockCanvas to pass null instead of rect, the performance is great at 60 frames per second, but the contents of the TextureView flicker and get corrupted, which is disappointing. I would think that it would just use the buffer or texture of the OpenGL frame under it, or at least be able to save the contents.

Are there any other options that don't do everything in raw OpenGL on Android for high-performance drawing and surface painting that persists between drawing calls?

+6
source share
2 answers

UPDATE I removed TextureView and now use the OpenGL view, where I call glTexSubImage2D to update the changed fragments of the rendering texture.

OLD ANSWER I ended up tiling a TextureView in a 4x4 grid. Depending on the dirty rectangle of each frame, I update the corresponding TextureView. Any view that doesn't update when I invoke Invalidate.

Some devices, such as the Moto G phone, have a problem where double buffering is corrupted for a single frame. You can fix this by calling lockCanvas twice when the parent view calls its onLayout.

 private void InvalidateRect(int l, int t, int r, int b) { dirtyRect.Set(l, t, r, b); foreach (DrawSubView v in drawViews) { if (Rect.Intersects(dirtyRect, v.Clip)) { v.RedrawAsync(); } else { v.Invalidate(); } } Invalidate(); } protected override void OnLayout(bool changed, int l, int t, int r, int b) { for (int i = 0; i < ChildCount; i++) { View v = GetChildAt(i); v.Layout(v.Left, v.Top, v.Right, v.Bottom); DrawSubView sv = v as DrawSubView; if (sv != null) { sv.RedrawAsync(); // why are we re-drawing you ask? because of double buffering bugs in Android :) PostDelayed(() => sv.RedrawAsync(), 50); } } } 
+1
source

First of all, if you want to understand what is happening under the hood, you need to read the Android Graphics graphic project . It's a long time, but if you sincerely want to understand why, this is the place to start.

About TextureView

TextureView works as follows: it has a Surface, which is a queue of buffers with producer-consumer relations. If you use software rendering (Canvas), you block Surface, which gives you a buffer; you rely on him; then you unblock the Surface, which sends the buffer to the consumer. The consumer in this case is in the same process and is called SurfaceTexture or (internally, more aptly) GLConsumer. It converts the buffer to an OpenGL ES texture, which is then displayed in the view.

If you disable hardware acceleration, GLES is disabled, and TextureView cannot do anything. That's why you got nothing when you turned off hardware acceleration. The documentation is very specific: "TextureView can only be used in a hardware accelerated window. When rendering in software, TextureView does not draw anything."

If you specify a dirty rectangle, the software rendering will memcpy the previous contents into the frame after rendering is complete. I don’t think it sets a direct clip, so if you call drawColor (), you fill the whole screen and then reload these pixels. If you are not currently installing a direct clip, you can see some performance benefits from this. (I did not check the code, though.)

A dirty rectangle is an input parameter. You pass in the required address when you call lockCanvas() , and the system is allowed to change it before the call returns. (In practice, the only reason this will be done is if the previous frame or surface size had not been changed, in which case he would expand it to cover the entire screen. I think it would be better processed with more direct β€œI reject your direct signal.”) You need to update every pixel inside the rectangle that you will return. You are not allowed to change the rectangle that you are trying to do in your example - everything that is in the dirty rectangle after lockCanvas() is done is what you need to draw.

I suspect that improper handling of dirty lines is the source of your flicker. Unfortunately, this is not a simple mistake, since the behavior of the lockCanvas() dirtyRect described only in the Surface class itself .

Surfaces and Buffering

All surfaces are double or triple buffers. There is no way around this - you cannot read and write at the same time, not break. If you need one buffer that you can change and click whenever you want, this buffer will need to be locked, copied and unlocked, which creates kiosks in the composition pipeline. For best throughput and latency, flip buffers are best.

If you want lock-unlock behavior, you can write it yourself (or find a library that does this) and it will be as effective as if the system did this for you (if you want, re good with blit loop). Draw a bitmap on the Canvas and blit screen, or in OpenGL ES FBO and blit buffer. You can find an example of the latter in Grafika's β€œ Record GL Application ” Activity, which has a mode that displays once off-screen and then blits twice (once for display, once for video recording).

Higher speed and

There are two main ways to draw pixels on Android: with Canvas or with OpenGL. Mapping the canvas to the surface or bitmap is always done in software, and OpenGL is rendered using the GPU. The only exception is that when rendering a custom view, you can use hardware acceleration, but this does not apply to rendering to a SurfaceView or TextureView.

A drawing or drawing application can either memorize drawing commands or simply throw pixels into the buffer and use this as its memory. The former allows you to β€œundo" deeper, the latter is much simpler and has higher productivity, as the amount of material for rendering is growing. It looks like you want to do the latter, so mixing from the screen does not make sense.

Most mobile devices have a hardware limit of 4096x4096 or less for GLES textures, so you cannot use a single texture for anything more. You can request a size limit (GL_MAX_TEXTURE_SIZE), but you might be better off with the internal buffer you need and just draw the part that fits the screen. I don't know what the Skia (Canvas) limitation is, but I believe that you can create much larger bitmaps.

Depending on your needs, SurfaceView may be preferable to TextureView, as it avoids the intermediate stage of the GLES texture. Everything that you paint on the surface is directly related to the system builder (SurfaceFlinger). The downside of this approach is that since the Surface user is not in the process, there is no way for the View system to process the output, so Surface is an independent layer. (For a drawing program, this can be useful - the image that is being drawn is on the same level, your user interface is on a separate layer at the top.)

FWIW, I didn’t look at the code, but Dan Sandler Markers application may be worth a peek (source code here ).

Update: corruption has been identified as a bug and fixed in 'L' .

+14
source

Source: https://habr.com/ru/post/989761/


All Articles