Minimizing the size of the "bitmap" canvas

Question

Context: multi-user application (node.js) - 1 artist, n clients

Canvas Size: 650x400 px (= 260,000 px)

In order for the canvas to be updated frequently (I think about 10 times per second), I need the data size to be as small as possible, especially when you think about download speed.

The toDataURL() method returning a base64 string is nice, but it contains a ton of data that I don't even need (23 bits per pixel). Its length is 8,088 (without previous MIME information) and assuming that JavaScript strings are 8-bit encoding, which will be 8.1 kilobytes of data, 10 times per second.

My next attempt was to use JS objects for various context actions, such as moveTo(x, y) or lineTo(x, y) , send them to the server and receive data in delta updates (through timestamps) by clients. However, this turned out to be even less efficient than the base64 string.

 { "timestamp": 0, "what": { "name": LINE_TO, "args": {"x": x, "y": y} } } 

This does not work smoothly or accurately, because there are already about 300 lineTo commands when you quickly brush. Sometimes there is no part of the movement (a straight line, not a rounded one), sometimes events are not even recognized by the client side of the script, because it seems to be "overloaded" with a mass of events that have already been triggered.

So, I have to finish using the base64 string with my 8.1 KB. I don’t want to worry about this - but even if done asynchronously with delta updates, there will be big delays on the real server, not to mention the random overflow of bandwidth.

The only colors I use are # 000 and #FFF, so I was thinking about a 1-bit data structure with only delta updates. That would be enough, and I would not mind any “color” precision losses (after all, it is black).

When most of the canvas is white, you might consider adding an extra Huffman length encoding to reduce the size even further. Like a canvas 50x2 px in size, and one black pixel in (26, 2) will return the following line: 75W1B74W (50 + 25 white pixels, then 1 black pixel, then another 24 white pixels)

It would even help if the canvas consisted of a 1-bit string like this:

 00000000000000000000000000000000000000000000000000 00000000000000000000000001000000000000000000000000 

This will help a lot.

My first question: how to write an algorithm to efficiently obtain this data?

Second: how can I pass pure canvas binary data to clients (via node server)? How do I even send a 1-bit data structure to the server? Should I convert my bits to a hexadecimal (or more) number and reanalyze?

Can this be used as a data structure?

Thanks in advance,

Harti

Answer 1

In the end, I figured it out. I used the algorithm to get image data within a specific area (i.e., the area currently drawn), and then paste the image data into the same coordinates.

• During drawing, I inform my application about how large the changed area is and where it starts (stored in currentDrawingCoords ).

• pixels is an ImageData array obtained by calling context.getImageData(left, top, width, height) with the saved coordinates of the drawing.

• getDeltaUpdate is called onmouseup (yes, this is a lack of scope idea):

   getDeltaUpdate = function(pixels, currentDrawingCoords) { var image = "" + currentDrawingCoords.left + "," + // x currentDrawingCoords.top + "," + // y (currentDrawingCoords.right - currentDrawingCoords.left) + "," + // width (currentDrawingCoords.bottom - currentDrawingCoords.top) + ""; // height var blk = 0, wht = 0, d = "|"; // http://stackoverflow.com/questions/667045/getpixel-from-html-canvas for (var i=0, n=pixels.length; i < n; i += 4) { if( pixels[i] > 0 || pixels[i+1] > 0 || pixels[i+2] > 0 || pixels[i+3] > 0 ) { // pixel is black if(wht > 0 || (i == 0 && wht == 0)) { image = image + d + wht; wht = 0; d = ","; } blk++; //console.log("Pixel " + i + " is BLACK (" + blk + "-th in a row)"); } else { // pixel is white if(blk > 0) { image = image + d + blk; blk = 0; d = ","; } wht++; //console.log("Pixel " + i + " is WHITE (" + blk + "-th in a row)"); } } return image; } 

• image is a string with a header part ( x,y,width,height|... ) and a part of the data body ( ...|w,b,w,b,w,[...] )

• The result is a string with fewer characters than the base64 string (unlike the 8k character string, there are 1k-6k characters in delta updates, depending on how many things were drawn in the modification area)

• This line is sent to the server, transferred to all other clients, and returned to ImageData using getImageData :

   getImageData = function(imagestring) { var data = imagestring.split("|"); var header = data[0].split(","); var body = data[1].split(","); var where = {"x": header[0], "y": header[1]}; var image = context.createImageData(header[2], header[3]); // create ImageData object (width, height) var currentpixel = 0, pos = 0, until = 0, alpha = 0, white = true; for(var i=0, n=body.length; i < n; i++) { var pixelamount = parseInt(body[i]); // amount of pixels with the same color in a row if(pixelamount > 0) { pos = (currentpixel * 4); until = pos + (pixelamount * 4); // exclude if(white) alpha = 0; else alpha = 255; while(pos < until) { image.data[pos] = 0; image.data[pos+1] = 0; image.data[pos+2] = 0; image.data[pos+3] = alpha; pos += 4; } currentpixel += pixelamount; white = (white ? false : true); } else { white = false; } } return {"image": image, "where": where}; } 

• Call context.putImageData(data.image, data.where.x, data.where.y); to put an area on top of everything there is!

As mentioned earlier, this may not be the perfect suit for every kind of monochrome canvas drawing application, since the modification area only sends onmouseup . However, I can live with this trade-off, because it is much less stressful for the server than all the other methods presented in the question.

I hope I was able to help people follow this issue.

Answer 2

I need to keep the data size as small as possible

Then do not send all the data. Only post changes close to what you offer yourself.

Make the framework so that each user can only perform "actions", such as "draw a black strokeWidth 2 from X1, Y1 to X2, Y2".

I would not worry about some kind of pure binary thing. If there are only two colors that are easy to send as the string "1,2, x, y, x2, y2" that other people will analyze just like the local client does, and it will be drawn in the same way.

I would not overdo it. Ask him to work with simple strings before worrying about any smart encoding. First, try a simple thing. Maybe the performance will be good without going through a lot of trouble!