Complex image segmentation: background and objects are similar

Question

I am new to image segmentation, but I need to do this to get the database for the machine learning classifier.

Essentially, I have a video similar to this image:

My task is to identify the cows in the foreground, or at least any cow. I understand that there is a problem with occlusion, but for the starter, I would like to correctly segment a lonely cow, for example, with a red rectangle around it (hand-drawn).

In less complex tasks like me, I distinguish by adding a threshold for each pixel, which either becomes (0,0,0) for the object, or (255,255,255) for the background:

Then I put pixels with the same values ​​to get classes and get a rectangle for large enough "blocks".

For the image above, this approach will not work, since objects and background are similar + there are a lot of shadows, side lighting, etc., so I'm not sure how to approach it. Any suggestions are welcome.

Answer 1

I understand that this is an old thread, but I would like to offer an approach to problems such as this.

You can try segmented texture, as the grassy background has a different texture from the cow.

Take a look at this link , where the characteristics of the texture energy for the image are determined in accordance with the technique of laws.

The law technique is implemented here in Python. It works by defining two-dimensional cores used to extract various objects in an image, such as edges, ripples, drops, and combinations thereof. The function below returns 9 images from which you can extract texture objects.

def laws(array): # Define the 1D kernels L5 = np.array([1,4,6,4,1]) # level E5 = np.array([-1,-2,0,2,1]) # edge S5 = np.array([-1,0,2,0,-1]) # spot R5 = np.array([1,-4,6,-4,1]) # ripples # Generate 2D kernels L5E5 = np.outer(L5,E5) E5L5 = np.outer(E5,L5) L5R5 = np.outer(L5,R5) R5L5 = np.outer(R5,L5) E5S5 = np.outer(E5,S5) S5E5 = np.outer(S5,E5) S5S5 = np.outer(S5,S5) R5R5 = np.outer(R5,R5) L5S5 = np.outer(L5,S5) S5L5 = np.outer(S5,L5) E5E5 = np.outer(E5,E5) E5R5 = np.outer(E5,R5) R5E5 = np.outer(R5,E5) S5R5 = np.outer(S5,R5) R5S5 = np.outer(R5,S5) return (0.5*(correlate(array, L5E5) + correlate(array, E5L5)), \ 0.5*(correlate(array, L5R5) + correlate(array, R5L5)), \ 0.5*(correlate(array, E5S5) + correlate(array, S5E5)), \ correlate(array, S5S5), \ correlate(array, R5R5), \ 0.5*(correlate(array, L5S5) + correlate(array, S5L5)), \ correlate(array, E5E5), \ 0.5*(correlate(array, E5R5) + correlate(array, R5E5)), \ 0.5*(correlate(array, R5S5) + correlate(array, S5R5))) 

Answer 2

I would try to take two photos. One photograph of a “static” background without cows, then a second photograph with cow (s). Then you can subtract these two images. I'm not too familiar with python, but imagemagick can easily distinguish between images ( http://www.imagemagick.org/Usage/compare/ ). Ideally, a "different" image isolates the cow (s). From there, you can get fancy and try various border detection algorithms, etc.

Hope this helps.