How to implement Optical Flow Tracker?

Question

I am using the OpenCV shell - Emgu CV, and I am trying to implement a motion tracker using Optical Flow, but I cannot figure out a way to combine horizontal and vertical information obtained from the OF algorithm:

flowx = new Image<Gray, float>(size); flowy = new Image<Gray, float>(size); OpticalFlow.LK(currImg, prevImg, new Size(15, 15), flowx, flowy); 

My problem is not how to combine vertical and horizontal motion information to build a tracker of moving objects? New image?

By the way, is there an easy way to display stream information in the current frame?

Thanks in advance.

Answer 1

Here is the feature that I defined in my YouTube motion tracking tutorial. . You can find the full source code attached to the video.

 void ComputeDenseOpticalFlow() { // Compute dense optical flow using Horn and Schunk algo velx = new Image<Gray, float>(faceGrayImage.Size); vely = new Image<Gray, float>(faceNextGrayImage.Size); OpticalFlow.HS(faceGrayImage, faceNextGrayImage, true, velx, vely, 0.1d, new MCvTermCriteria(100)); #region Dense Optical Flow Drawing Size winSize = new Size(10, 10); vectorFieldX = (int)Math.Round((double)faceGrayImage.Width / winSize.Width); vectorFieldY = (int)Math.Round((double)faceGrayImage.Height / winSize.Height); sumVectorFieldX = 0f; sumVectorFieldY = 0f; vectorField = new PointF[vectorFieldX][]; for (int i = 0; i < vectorFieldX; i++) { vectorField[i] = new PointF[vectorFieldY]; for (int j = 0; j < vectorFieldY; j++) { Gray velx_gray = velx[j * winSize.Width, i * winSize.Width]; float velx_float = (float)velx_gray.Intensity; Gray vely_gray = vely[j * winSize.Height, i * winSize.Height]; float vely_float = (float)vely_gray.Intensity; sumVectorFieldX += velx_float; sumVectorFieldY += vely_float; vectorField[i][j] = new PointF(velx_float, vely_float); Cross2DF cr = new Cross2DF( new PointF((i*winSize.Width) +trackingArea.X, (j*winSize.Height)+trackingArea.Y), 1, 1); opticalFlowFrame.Draw(cr, new Bgr(Color.Red), 1); LineSegment2D ci = new LineSegment2D( new Point((i*winSize.Width)+trackingArea.X, (j * winSize.Height)+trackingArea.Y), new Point((int)((i * winSize.Width) + trackingArea.X + velx_float), (int)((j * winSize.Height) + trackingArea.Y + vely_float))); opticalFlowFrame.Draw(ci, new Bgr(Color.Yellow), 1); } } #endregion } 

Answer 2

Visualization of optical flow . A general approach is to use the color field of a 2D stream. This means that we display the stream as an image, where the pixel intensity corresponds to the absolute value of the stream in the pixel, and the hue reflects the direction of the stream. Look at the pic. 2 in [ Baker et al., 2009 ]. Another way is to draw the flow vectors in the grid above the first image (say, every 10 pixels).

The union of x and y . It is unclear what you mean here. The pixel (x, y) from the first image moves to (x + flowx, y + flowy) on the second. Thus, in order to track the object, you fix the position of the object in the first image and add the value of the stream to get its position in the second.

Answer 3

There are some well-known optical flow algorithms. one of them that may be useful to you is Lucas Kanade .. you can find the matlab source here