Android.grapics.matrix for translating OpenGL 2.0 ES textures

Question

Currently, the application displays an ImageView that successfully scales and pans on the screen. In addition to this image, I would like to provide a texture that I would like to update to scale or pan when the image below it zooms or pans.

As I understand it, this should be possible using getImageMatrix () of my current setting in ImageView and then applying this to a textured bitmap that sits on top of the original image.

Editing and Resolution: (with a strong assistant from the selected answer below)

Currently, texture panning occurs at a different speed than ImageView, but when it was enabled, I update this publication with additional changes and provide a solution for the entire application. Then only the solution and a brief description of the problem description will remain. Perhaps I will even publish the source code for the surface and rendering to use a scalable surface view.

To map ImageView to OpenGL texture, you had to do a couple of things to get it right. And hopefully this will help other people who may want to use the scalable SurfaceView in the future.

Below is the shader code. Gl_FragColor takes a 3x3 translation matrix from the getImageMatrix of any ImageView to refresh the screen using the onDraw () method.

private final String vertexShader_ = "attribute vec4 a_position;\n" + "attribute vec4 a_texCoord;\n" + "varying vec2 v_texCoord;\n" + "void main() {\n" + " gl_Position = a_position;\n" + " v_texCoord = a_texCoord.xy;\n" + "}\n"; private final String fragmentShader_ = "precision mediump float;\n" + "varying vec2 v_texCoord;\n" + "uniform sampler2D texture;\n" + "uniform mat3 transform;\n" + "void main() {\n" + " vec2 uv = (transform * vec3(v_texCoord, 1.0)).xy;\n" + " gl_FragColor = texture2D( texture, uv );\n" + "}\n"; 

In the beginning, the transformation matrix for OpenGL is just an identification matrix, and our ImageView will update these common values ​​using a shader through a listener.

 private float[] translationMatrix = {1.0f, 0.0f, 0.0f, 0.0f, 1.0f, 0.0f, 0.0f, 0.0f, 1.0f}; ... uniforms_[TRANSLATION_UNIFORM] = glGetUniformLocation(program_, "transform"); checkGlError("glGetUniformLocation transform"); if (uniforms_[TRANSLATION_UNIFORM] == -1) { throw new RuntimeException("Could not get uniform location for transform"); } .... glUniformMatrix3fv(uniforms_[TRANSLATION_UNIFORM], 1, false, FloatBuffer.wrap(translationMatrix)); .... 

However, the OpenGL code requires an inverse calculation of the OpenGL matrix that comes from Android, as well as the transpose operation that is performed on them before they are passed to the visualizer for display on the screen. This is due to how this information is stored.

 protected void onMatrixChanged() { //... float[] matrixValues = new float[9]; Matrix imageViewMatrix = getImageViewMatrix(); Matrix invertedMatrix = new Matrix(); imageViewMatrix.invert(invertedMatrix); invertedMatrix.getValues(matrixValues); transpose(matrixValues); matrixChangedListener.onTranslation(matrixValues); //... } 

In addition to the fact that the values ​​are in the wrong locations for input in OpenGL rendering, we also have a problem that we are dealing with our translations on the imageHeight and imageWidth scales instead of the normalized [0, 1] that OpenGL expects. So, to fix this, we need to target the last column with divisible numbers of our width and height.

 matrixValues[6] = matrixValues[6] / getImageWidth(); matrixValues[7] = matrixValues[7] / getImageHeight(); 

Then, after you do this, you can start mapping using the ImageView matrix functions or the updated ImageTouchView matrix functions (which have some additional nice features when handling images on Android).

The effect can be seen below when the ImageView behind it occupies the entire screen, and the texture in front of it is updated based on updates from ImageView. It handles the scaling and panning of translations between them.

The only other step in this process would be to resize the SurfaceView to fit the image in ImageView to get a full screen when the user approaches. To get only the displayed image, you can simply put an “invisible” image in the background. And just let ImageTouchView updates change the way your OpenGL SurfaceView displays.

Run the answer below , as they helped me offline via chat to understand what needs to be done to link the image to the texture. They deserve it.

If you end up moving to modifying the vertex shader below, instead of resolving the fragment shader, then it seems that the values ​​no longer need to be rebuilt to work with larger zoom (and actually work in the opposite direction if you leave it as it is) . In addition, panning seems to work in the opposite direction than expected.

Answer 1

To do this, you can fake a fragment shader, just set this matrix as a single parameter of the shader fragment and change the texture coordinates using this matrix.

 precision mediump float; uniform sampler2D samp; uniform mat3 transform; varying vec2 texCoord; void main() { vec2 uv = (transform * vec3(texCoord, 1.0)).xy; gl_FragColor = texture2D(samp, uv); } 

Note that the matrix you get from the image view is a string order that you must convert to get the general order of the OpenGL columns, also you would separate the translation components x and y by width and height, respectively.

Update . It just seemed to me that changing the vertex shader would be better, you can just scale your quad with the same matrix that you used in the fragment shader, and it will be clamped on your screen, if to big, and it will not have a problem with rounding the edge at small, and matrix multiplication will occur only 4 times instead of one for each pixel. Just execute in the vertex shader:

 gl_Position = vec4((transform * vec3(a_position.xy, 1.0)).xy, 0.0, 1.0); 

and use a non-transformed tex encoder in fragment shaders.