Extract object from image of box with object

Question

I have a box transparent in front and I place the camera on the front transparent panel to capture the image of the inside, most of the time the box is empty, but suppose someone puts an object inside this window, then I just need to extract this object from the captured Images.

(My real goal is to recognize the object placed inside the field, but the first step is to extract the object and then extract the functions to create the training model, and so far I am only focused on extracting the object from the image)

I am new to OpenCV and use it with Python, and I found several OpenCV features that can help me.

• GrabCut, this works fine for me, I can just extract the object, provided that I mark the rectangle above the object, but how the object can be anywhere inside the box, so the exact rectangle of the size of the object is impossible, and if there is a way, please suggest me .
• Image difference, since I have an empty image in the cavity and when the object is present, I can use the cv2.absdiff function to calculate the difference between the image, but this does not work properly in most cases, since it uses pixel difference and pixel calculations for this, the results are strange, plus changing lighting conditions also makes work difficult.
• Back Ground Subtraction, I read a few posts about it, and it looks like this: what I need, but the example I got for the video, and I don’t understand how to make it work with two images, one empty field and the other with the object.

The code for subtracting the back surface is as follows, even if it does not work properly for short distances

cap = cv2.VideoCapture(0)
fgbg = cv2.createBackgroundSubtractorMOG2()
fgbg2 = cv2.createBackgroundSubtractorKNN()

while True:
    ret, frame = cap.read()
    cv2.namedWindow('Real', cv2.WINDOW_NORMAL)
    cv2.namedWindow('MOG2', cv2.WINDOW_NORMAL)
    cv2.namedWindow('KNN', cv2.WINDOW_NORMAL)
    cv2.namedWindow('MOG2_ERODE', cv2.WINDOW_NORMAL)
    cv2.namedWindow('KNN_ERODE', cv2.WINDOW_NORMAL)
    cv2.imshow('Real', frame)
    fgmask = fgbg.apply(frame)
    fgmask2 = fgbg2.apply(frame)
    kernel = np.ones((3,3), np.uint8)
    fgmask_erode = cv2.erode(fgmask,kernel,iterations = 1)
    fgmask2_erode = cv2.erode(fgmask2,kernel,iterations = 1)

    cv2.imshow('MOG2',fgmask)
    cv2.imshow('KNN',fgmask2)
    cv2.imshow('MOG2_ERODE',fgmask_erode)
    cv2.imshow('KNN_ERODE',fgmask2_erode)
    k = cv2.waitKey(30) & 0xff
    if k == 27:
        break
cap.release()
cv2.destroyAllWindows()

Can anyone help on this topic, as well as how to modify the above code to just use two images when I tried to get blank images. Thanks at Advance

Sample images from the camera are as follows: (I use an 8 megapixel camera, so the image size is large, I reduced the size and then uploaded it here)

Answer 1

, , . , , . . , - .

:

:

:

№1:

№2:

:

import cv2
import numpy as np

# load the images
empty = cv2.imread("empty.jpg")
full = cv2.imread("full_2.jpg")

# save color copy for visualization
full_c = full.copy()

# convert to grayscale
empty_g = cv2.cvtColor(empty, cv2.COLOR_BGR2GRAY)
full_g = cv2.cvtColor(full, cv2.COLOR_BGR2GRAY)

# blur to account for small camera movement
# you could try if maybe different values will maybe
# more reliable for broader cases
empty_g = cv2.GaussianBlur(empty_g, (41, 41), 0)
full_g = cv2.GaussianBlur(full_g, (41, 41), 0)

# get the difference between full and empty box
diff = full_g - empty_g
cv2.imwrite("diff.jpg", diff)

# inverse thresholding to change every pixel above 190
# to black (that means without the bag)
_, diff_th = cv2.threshold(diff, 190, 255, 1)
cv2.imwrite("diff_th.jpg", diff_th)

# combine the difference image and the inverse threshold
# will give us just the bag
bag = cv2.bitwise_and(diff, diff_th, None)
cv2.imwrite("just_the_bag.jpg", bag)

# threshold to get the mask instead of gray pixels
_, bag = cv2.threshold(bag, 100, 255, 0)

# dilate to account for the blurring in the beginning
kernel = np.ones((15, 15), np.uint8)
bag = cv2.dilate(bag, kernel, iterations=1)

# find contours, sort and draw the biggest one
_, contours, _ = cv2.findContours(bag, cv2.RETR_TREE,
                                  cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key=cv2.contourArea, reverse=True)[:3]
cv2.drawContours(full_c, [contours[0]], -1, (0, 255, 0), 3)

# show and save the result
cv2.imshow("bag", full_c)
cv2.imwrite("result2.jpg", full_c)
cv2.waitKey(0)

, , , . , , , , . , , , , , - .

GrabCut, JeruLuke , . , , .

Answer 2

. , .

cv2.Canny() :

gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY) #---convert image to gray---
blur = cv2.GaussianBlur(gray, (5, 5), 0)   #---blurred the image---
edges = cv2.Canny(blur, lower, upper)   #---how to find perfect edges see link below---

, :

kernel = np.ones((3, 3), np.uint8)
dilated = cv2.morphologyEx(edges, cv2.MORPH_DILATE, kernel)

, , .

_, contours, hierarchy = cv2.findContours(king, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)  

. cv2.RETR_EXTERNAL

, , , .

GrabCut Algorithm, .

I, GrabCut

:

, , , .

, . , ! : D