Training algorithm for preparing this data

Question

I work in MATLAB

Land

NOTE This shows the x - position pixel at the position (i,j) of the FIRST frame in all frames. This means that the pixel in (23.87) in the first frame has at the end of the sequence x-position as 35 (as seen in the graph).

Here are some typical x_pos for some different values (i,j) . (i,j) refers to a pixel in (i,j) in the first frame not in all frames

For (i,j) = (23 ,87)

(i,j) = (42 ,56)

(i,j) = (67 ,19)

Answer 1

A video is like a sequence of photographs of real objects.
And the real object, in front of the camera, can only do two different things:

• they stand still
• they move

If the pixel you are trying to predict is from a video, you need to see how the pixel moves around the screen because the object moves around the screen.

And this is how video compression works (H264, H265 ...) (obviously, the video compression algorithm is much more complicated, just try to understand the direction of the pixel ... :-))

Here are a few stackoverflow questions / answers that may help you:

• Motion vector calculation
• Kalman filter in computer vision: choice of covariances Q and R
• How to track object movement using video?
• Vehicle Segmentation and Tracking

Answer 2

This means that we are not talking about pixels in the image, but about a moving object, which makes the task more convenient. Your data is indeed a time series, so time-based algorithms are preferred. Their classic are Markov models (in particular , Markov chains and somewhat more complex hidden Markov models ).

However, your input is noisy due to camera instability. Thus, even a better solution would be to use a Kalman filter - a model similar to HMM, but with a clear understanding of noise. It is widely used in robotics, navigation, and similar fields to estimate the current and predict the future location of a vehicle based on inaccurate sensor data and historical information. Doesn't that look like what you need?

I don't really like Matlab, but it seems to have a kalman function that implements the mentioned filter.