OpenCV time for contact with the optical stream

Question

I am trying to write a program with C ++ and OpenCV that calculates the remaining time (frames) before colliding with an object. As an example of a video, I have a camera moving towards a board.

My approach to this is as follows:

• detect tracking functions (tried goodFeaturesToTrack () or setpoints "manually")
• calculate optical flow through calcOpticalFlowPyrLK ()
• calculate the fundamental matrix of previous and current found functions via findFundamentalMat ()
• check the correctness of the main matrix.
• calculate epipolar lines and epipolar - focus expansion in the video

Finally, I planned to use the time to contact the method to calculate the remaining frames before the collision.

So far, my biggest problem has been to find the right fundamental matrix and therefore the epipole. The calculated matrix seems to be irregular, as are the epipolar lines. Will my planned approach be right? Does anyone have a working example of getting FoE from a video or any kind of step-by-step instructions on what I should do?

I would be very happy for any help!

Thanks!

Answer 1

I think that LK optical stream will only detect 2D stream. Although in principle you can derive information from movement in a part of the image (that is, in the first quadrant), the fundamental matrix needs a stereo pair of images, not consecutive ones.

Above my head, I do not know the ideal steps to solve your problem. But I think that researching the structure from motion can help you, because this method solves the problem of creating three-dimensional data from motion, but without a pair of stereo cameras.