Mjpeg streaming in Nginx with low client-side bandwidth

Question

I am passing MJPEG using Nginx. This works fine as long as the client-side bandwidth is sufficient. When the bandwidth is insufficient, it seems to fall about 2 minutes behind, then it goes to the current frame and starts falling back again.

Is there a way to control the buffer so that it never stores more than 2 frames? “So, if the client side cannot keep up, will it never fall for more than a second or two?”

EDIT: basically the server (currently the python tornado behind the nginx reverse proxy) sends a 5 Mb stream, the client has a width of 1 Mb (for an argument) - the server (nginx or python) should be able to detect this and drop frames. The question is how?

Answer 1

It depends a lot on how you actually deploy the M-JPEG frames, and if you have to use the built-in browser support or you can write your own javascript.

Background

Keep in mind that when streaming M-JPEG from the server, it simply sends a number of JPEG files, but as a response to a single web request. That is, a regular web request looks like

Client             Server
   | --- Request ---> |
   |                  |
   | <-- JPEG File -- |

So far, the M-JPEG request is more like

Client             Server
   | --- Request ---> |
   |                  |
   | <- JPEG part 1 - |
   | <- JPEG part 2 - |
   | <- JPEG part 3 - |

, , , M-JPEG , , .

Pure JS Solution

javascript , , / . javascript ( JPEG). javascript ,

• . 50% ? .
• . 25%? 50%.

-, javascript , TCP-. Keep-Alive , Spdy HTTP/2 , Nginx, javascript. , javascript , , .

( jQuery imgload plugin ):

var timeout = 250; // 4 frames per second, adjust as necessary
var image = // A reference to the <img> tag for display
var accumulatedError = 0; // How late we are

var doFrame = function(frameId) {
    var loaded = false, timedOut = false, startTime = (new Date()).getTime();
    $(image).bind("load", function(e) {
        var tardiness = (new Date()).getTime() - startTime - timeout;
        accumulatedError += tardiness; // Add or subtract tardiness
        accumulatedError = Math.max(accumulatedError, 0); // but never negative
        if (!timedOut) {
            loaded = true;
        } else {
            doFrame(frameId + 1);
        }
    }
    var timeCallback = function() {
        if (loaded) {
            doFrame(frameId + 1); // Just do the next frame, we're on time
        } else {
            timedOut = true;
        }
    }
    while(accumulatedError > timeout) {
        // If we've accumulated more than 1 frame or error
        // skip a frame
        frameId += 1;
        accumulatedError -= timeout;
    }
    // Load the image
    $(image).src = "http://example.com/images/frame-" + frameId + ".jpg";
    // Start the display timer
    setTimeout(timeCallback, timeout);
}

doFrame(1); // Start the process

, , , , (, ).

-

javascript , , , . , 4 , , 250 , 250 . , . , , , , , - TCP ..

, (. ), Websockets . , Nginx -.

-. jpeg- , (, 30 20-25 , , , , , ). , , .

, , accumulatedError , , ( ). accumulatedError (, , ), reset accumulatedError.

, , , , , . - PID PID loop, , PID, , , accumulatedErrror - ( ) .