Image processing and texture mapping with HTML5 Canvas?

In the 3D engine I'm working on, I managed to successfully draw a cube in 3D. The only way to fill the sides is to use a solid color or gradient, as far as I know. To make things more exciting, I would really like to implement texture mapping with a simple bitmap.

The fact is, I can hardly find any articles or code examples regarding image processing in JavaScript. Moreover, support for images in HTML5 canvas seems to be limited by cropping.

How can I stretch a bitmap so that a rectangular bitmap can fill the irregular face of the cube? In 2D, the designed surface of a square cube is, because of the perspective, not a square shape, so I have to stretch it to fit in any quadrangle.

I hope this image will clarify my idea. The left face is now filled with a white / black gradient. How could I fill it with a bitmap after it was mapped?

Cube

Does anyone have any advice regarding perspective texture mapping (or image manipulation in general) using JavaScript and HTML5 canvas?

Edit: I got a job thanks to 6502!

It is, however, a fairly intensive processor, so I would like to hear any optimization ideas.

Result Using Method 6502 - Texture Image Used

+15
javascript html5 html5-canvas 3d texture-mapping
Jan 23 2018-11-11T00:
source share
1 answer

I think you will never get the exact result ... I spent some time learning how to do 3D graphics using the canvas 2d context, and I found it appropriate to do the texturing with gouraud shading, calculating the corresponding 2d gradients and matrices:

  • Solid polygons are, of course, light.
  • Gouraud filling is possible only on one component (i.e. you cannot have a triangle where each vertex is an arbitrary RGB filled with bilinear interpolation, but you can do this filling using, for example, three arbitrary shades of the same color)
  • Linear texture mapping can be done by cropping and drawing images

I would implement perspective texture mapping using mesh division (e.g. on PS1).

However, I found a lot of problems ... for example, drawing images with matrix transformation (necessary for texture mapping) is pretty inaccurate on chrome and IMO it is impossible to get the result with an exact pixel; in general, there is no way to turn off anti-aliasing when drawing on canvas, and this means that you will get visible through lines when split into triangles. I also found that multiprocessing rendering works very poorly on chrome (probably due to the way hw-accellerated rendering is implemented).

In general, this type of rendering is certainly a stress for web browsers and, apparently, these use cases (for example, strange matrices) are not very well tested. I was even able to make Firefox crash so badly that it destroyed the entire X susbsystem system on my Ubuntu.

You can see the results of my efforts here or as a video here ... IMO, it is undoubtedly impressive that this can be done in a browser without using 3D extensions, but I do not think that the current problems will be fixed in the future.

In any case, the main idea used to draw the image so that the 4 corners end in a certain position of the pixels is to draw two triangles, each of which will use bilinear interpolation.

In the following code, I assume that you have a texture image object and 4 angles, each of which is an object with fields x,y,u,v , where x,y are the pixel coordinates on the target canvas, and u,v are the pixel coordinates on texture :

 function textureMap(ctx, texture, pts) { var tris = [[0, 1, 2], [2, 3, 0]]; // Split in two triangles for (var t=0; t<2; t++) { var pp = tris[t]; var x0 = pts[pp[0]].x, x1 = pts[pp[1]].x, x2 = pts[pp[2]].x; var y0 = pts[pp[0]].y, y1 = pts[pp[1]].y, y2 = pts[pp[2]].y; var u0 = pts[pp[0]].u, u1 = pts[pp[1]].u, u2 = pts[pp[2]].u; var v0 = pts[pp[0]].v, v1 = pts[pp[1]].v, v2 = pts[pp[2]].v; // Set clipping area so that only pixels inside the triangle will // be affected by the image drawing operation ctx.save(); ctx.beginPath(); ctx.moveTo(x0, y0); ctx.lineTo(x1, y1); ctx.lineTo(x2, y2); ctx.closePath(); ctx.clip(); // Compute matrix transform var delta = u0*v1 + v0*u2 + u1*v2 - v1*u2 - v0*u1 - u0*v2; var delta_a = x0*v1 + v0*x2 + x1*v2 - v1*x2 - v0*x1 - x0*v2; var delta_b = u0*x1 + x0*u2 + u1*x2 - x1*u2 - x0*u1 - u0*x2; var delta_c = u0*v1*x2 + v0*x1*u2 + x0*u1*v2 - x0*v1*u2 - v0*u1*x2 - u0*x1*v2; var delta_d = y0*v1 + v0*y2 + y1*v2 - v1*y2 - v0*y1 - y0*v2; var delta_e = u0*y1 + y0*u2 + u1*y2 - y1*u2 - y0*u1 - u0*y2; var delta_f = u0*v1*y2 + v0*y1*u2 + y0*u1*v2 - y0*v1*u2 - v0*u1*y2 - u0*y1*v2; // Draw the transformed image ctx.transform(delta_a/delta, delta_d/delta, delta_b/delta, delta_e/delta, delta_c/delta, delta_f/delta); ctx.drawImage(texture, 0, 0); ctx.restore(); } } 

These ugly weird formulas for all these โ€œdelta variablesโ€ are used to solve two linear systems of three equations with three unknowns using the Cramer and Sarrus methods for 3x3 determinants.

In particular, we look for the values a , b , ... f so that the following equations are satisfied:

 a*u0 + b*v0 + c = x0 a*u1 + b*v1 + c = x1 a*u2 + b*v2 + c = x2 d*u0 + e*v0 + f = y0 d*u1 + e*v1 + f = y1 d*u2 + e*v2 + f = y2 

delta is a determinant of a matrix

 u0 v0 1 u1 v1 1 u2 v2 1 

and, for example, delta_a is the determinant of the same matrix when replacing the first column with x0 , x1 , x2 . With their help, you can calculate a = delta_a / delta .

+37
Jan 23 2018-11-11T00:
source share
โ€” -



All Articles