How to determine the distance from the object in the video?

Question

I have a video file recorded on the front side of a moving car. I am going to use OpenCV to detect and recognize objects, but I stick to one aspect. How to determine the distance from a recognized object.

I can find out my current speed and real GPS position, but that’s all. I cannot make any assumptions about the object that I am tracking. I plan to use this to track and track objects without encountering them. Ideally, I would like to use this data to get the object real position, which I could do if I could determine the distance from the camera to the object.

Answer 1

When you move a video, you can use time parallax to determine the relative distance of objects. Parallax: ( definition ).

The effect will be the same as what we get with our eyes, which can get a perception of depth by looking at the same object from slightly different angles. As you move, you can use two consecutive video frames to get a slightly different angle.

Using parallax calculations, you can determine the relative size and distance of objects (relative to each other). But, if you want absolute size and distance, you will need a known reference point.

You also need to know the speed and direction of movement (as well as the frame rate of the video) in order to perform the calculations. You can get vehicle speed using visual data, but this adds another dimension to complexity.

The technology already exists. Satellites determine topographic fame (altitude) by comparing several images taken in a short period of time. We use parallax to determine the distance of stars by photographing the night sky at different points in the earth's orbit around the sun. I managed to create three-dimensional images from the airplane window by taking two photos in a short time.

Exact technology and calculations (even if I knew them from head to toe) were outside the scope of discussion here. If I find a decent link, I will post it here.

Answer 2

Your problem is quite standard in this area.

Firstly,

you need to calibrate the camera. This can be done offline (simplifies life significantly ) or online through self-calibration.

Refuse it offline - please.

Secondly,

Once you have the camera calibration matrix K, determine the camera projection matrix in a sequential scene (you need to use parallax, as others have mentioned). This is well described in this OpenCV tutorial .

You will need to use GPS information to find the relative orientation between the cameras in successive scenes (which can be problematic due to the noise inherent in most GPS units), i.e. the R and t mentioned in the tutorial or the rotation and translation between the two cameras.

Once you solve all this, you will have two projection matrices --- camera views in these consecutive scenes. Using one of these so-called camera matrices, you can "project" a 3D point M onto the scene onto a 2D image of the camera onto the pixel coordinate m (as in the tutorial).

We will use this to triangulate a real three-dimensional point from the 2D points found in your video.

Thirdly,

use a percentage point detector to track the same point in your video that’s on the object of interest. Several detectors are available, I recommend SURF since you have OpenCV, which also has several other detectors, such as Shi-Tomasi Corners , Harris , etc. .

Fourth,

Once you track the points of your object in sequence and get the corresponding coordinates of the 2D pixel, you should triangulate for the best 3D fit considering your projection matrix and two-dimensional points.

The above image perfectly reflects the uncertainty and how the optimal 3D point is calculated. Of course, in your case, the cameras are probably ahead of each other!

Finally,

Once you have the 3D objects of the object, you can easily calculate the Euclidean distance between the center of the camera (which is the source in most cases) and the point.

Note

This is obviously not easy, but it is not so difficult. I recommend Hartley and Zisserman the excellent book Multiple View Geometry , which described everything in detail above, using the MATLAB code to download.

Good luck and keep asking questions!

Answer 3

You need to identify the same points in the same object on two different frames taken at a known distance from each other. Since you know the location of the camera in each frame, you have a base level (a vector between two camera positions. Draw a triangle from a known baseline and angles to identified points. Trigonometry gives you the length of the unknown sides of the trail for a known baseline length and the known angles between the baseline line and unknown parties.

You can use two cameras or one camera with consecutive shots. So, if your car moves at a speed of 1 m / s, and you take in celebrities every second, then the frames will successfully give you a 1-meter base level, which should be good for measuring the distance to an object, say, at a distance of 5 meters. If you need to clean objects further than the frames used, you need to do even more - however, more distant objects will be displayed longer.

The observer in F1 sees the target at T with an angle a1 to the velocity vector. The observer moves the distance b to F2. He sees the target at T with an angle a2.

Find r1, range from target in F1

The trigonometric identity for cosine gives

Cos (90 - a1) = x / r1 = c1

Cos (90 - a2) = x / r2 = c2

Cos (a1) = (b + z) / r1 = c3

Cos (a2) = z / r2 = c4

x is the distance to the target orthogonal to the speed vector of the observers

z is the distance from F2 to the intersection with x

Solution for r1

r1 = b / (c3 - c1. c4 / c2)

Answer 4

Two cameras so you can detect parallax. This is what people do.

change

See ravenspoint answer for more details. Also, keep in mind that a single camera with a splitter is sufficient.

Answer 5

Use stereo imaging cards. many implementations afloat, here are a few links: http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_COPIES/OWENS/LECT11/node4.html

http://www.ece.ucsb.edu/~manj/ece181bS04/L14(morestereo).pdf

In this case, you do not have a stereo camera, but the depth can be estimated using the video http://www.springerlink.com/content/g0n11713444148l2/

I think the above will be what can help you the most.

studies have progressed so far that depth can be estimated (although not satisfactorily) from a single monocular image http://www.cs.cornell.edu/~asaxena/learningdepth/

Answer 6

Someone, please correct me if I am mistaken, but it seems to me that if you are going to just use one camera and just rely on a software solution, any processing that you could do would be prone to false positives. I doubt very much that there is any processing that could distinguish between objects that are really at a perceived distance and those that only appear at that distance (for example, “forced perspective”) in films.

Can I add an ultrasonic sensor?

Answer 7

Place an object of a known size in the field of view of the cameras. This way you can have a more objective metric for measuring angular distances. Without a second viewpoint / camera, you will be limited to an estimate of size / distance, but at least this will not be a complete assumption.

Answer 8

you must first calibrate your camera so that you can establish a relationship between the positions of objects in the camera plan and their positions in the real world plan if you use one camera, so you can use the “optical flow method” if you use two cameras, you just use a simple triangle to find the real position (it will be easy for you to find the distance from the objects), but the trial with this second metosis is a coincidence, which means how you can find the position of the object 'x' in camera2 if you already set it to camera1, and here you can use the SIFT algorithm. I just gave you some keywords that could help you.