The dominant color in the video for each frame

Question

I want to analyze video (mp4) when the LED is blinking. The background is usually gray, but the color of the LED may change. The LED is close enough in the video, so the LED indicator is the largest part of the frame. I found Color-Thief, but this is just for images, not for videos. Since the flashing frequency may also vary, I need to check every frame of the video for the dominant color.

If anyone has any ideas, I would really appreciate help. Thanks in advance for your time.

EDIT: Two screenshots of the video (the first red LED is off, the second red LED is on) (I had to remove the link for the color thief for two links to the screenshot)

This is a function that is called when the video starts and should analyze the video. I tried to do this through a medium color, but it is useless, because usually it is some kind of gray / brown.

function processFrame(e) { var video = document.getElementById("video"); if (video.ended){ //dauer(); console.log("Ende"); //alert("Ende"); } if (video.paused || video.ended) { return; } var bufferCanvas = document.getElementById("buffer"); var displayCanvas = document.getElementById("display"); var buffer = bufferCanvas.getContext("2d"); var display = displayCanvas.getContext("2d"); buffer.drawImage(video, 0,0, bufferCanvas.width, displayCanvas.height); var frame = buffer.getImageData(sx, sy, sw, sh); var length = frame.data.length / 4; for (var i = 0; i < length; i++) { var r = frame.data[i * 4 + 0]; var g = frame.data[i * 4 + 1]; var b = frame.data[i * 4 + 2]; average= average +((r+g+b)/3) } averagecolor[i]=average display.putImageData(frame, 0, 0); setTimeout(processFrame, 0); } 

Answer 1

This is a bit broad as you ask, since you need to do this in a few steps - but it is quite possible to do so. To reduce the time, I will simply present the steps you need to take in order to be able to analyze the LEDs.

You can do this with fully automatic or semi-automatic predefined areas in the video frame.

The auto-detection steps are as follows and are used once per session in case of camera movement or zooming. The lighting condition may also affect the detection, but here I assume that the lighting conditions are the same, as well as the images that you display. It is also specific to these images and may not work if you change the settings of the routers / camera.

Initial step to automatically detect areas:

• Scan the horizontal lines halfway and start registering regions when all the values ​​are RGB = 255 (this is the white center of the LEDs). Set the flag that is in the region and count the number of pixels at the start of registration based on the threshold (for example, 7-10 pixels, which should be white in a row). This means that individual pixels will be white.
• If you can’t find the white pixels at the end of the line (based on the threshold) and prepare for a new registration.
• When the bottom of the first half (I set the maximum height shorted than the actual one), start scanning from the middle to the right edge.

After that, you should have six regions registered (use the "calibration" frame where all the LEDs are on - this can be started manually before starting live).

For the next step, which will work continuously, you will use these regions as snap areas and select points on only one side and only a few pixels in height.

• Use the region as an anchor and define a selection region based on the offset x + the width of the region + 2 pixels as the start of X (this is just a suggestion for the initial offset - you may need to fine-tune it). Slide Y + 10 and set the width and height to 10 pixels. This should give you a 100-point sample area right next to the LED.
• Run statistics on this small region with getImageData. You need to check all three components, since you need to filter the light gray color. Find the threshold value (for example, if r > 230 && b < 200 && g < 200 only to take some values) when you accumulate your own red value (and the other green). This process requires a calibration phase to find this threshold value that you need to use. Run some tests. If the camera changes f-stop, you will probably need to recalibrate.
• Accumulate green and red values ​​for different variables. You do not need to average them, but you need to find the threshold value for the total, where you know that the LED is on for the LED with this color (for this, also use the initial test frame from the area detection area). Add some tolerance to the values ​​by comparing the regions when the LEDs are off.

This will give you just 600 points to scan, possibly less if you can fine tune it.

For the manual method, you skip setting the area as described in the first step, but use the frame and manually set the areas, measuring the pixels of the white top. If you change the position of the camera, you will need to do this.

As you understand, there is a small code that needs to be written for this, and there are many factors that will affect the final code, which makes it very specific for this use. And I believe that this is partly beyond the scope of SO, but I hope this gives you a few reasons to get you started.