Capturing screen video in Python

Question

Is there a way with Python (possibly with OpenCV or PIL) to continuously capture frames of all or parts of the screen, at least at 15 frames per second or more? I saw this in other languages, so theoretically this should be possible.

I do not need to save image data to a file. I just want it to output an array containing raw RGB data (for example, in a numpy array or something else), since I'm going to just take it and send it to a large LED display (maybe after recalibrating it).

Answer 1

There is another solution with mss that provides a much better frame rate. (Tested on Macbook Pro with MacOS Sierra)

import numpy as np import cv2 from mss import mss from PIL import Image mon = {'top': 160, 'left': 160, 'width': 200, 'height': 200} sct = mss() while 1: sct.get_pixels(mon) img = Image.frombytes('RGB', (sct.width, sct.height), sct.image) cv2.imshow('test', np.array(img)) if cv2.waitKey(25) & 0xFF == ord('q'): cv2.destroyAllWindows() break 

Answer 2

You will need to use ImageGrab from the Pillow library (PIL) and convert the capture to a numpy array. When you have an array, you can do what you like using opencv. I converted the capture to gray and used imshow () as a demo.

Here is a quick code to get you started:

 from PIL import ImageGrab import numpy as np import cv2 img = ImageGrab.grab(bbox=(100,10,400,780)) #bbox specifies specific region (bbox= x,y,width,height *starts top-left) img_np = np.array(img) #this is the array obtained from conversion frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2GRAY) cv2.imshow("test", frame) cv2.waitKey(0) cv2.destroyAllWindows() 

you can connect the array there with the frequency you want to save frames. After that, you simply decode the frames. don't forget to add before the loop:

 fourcc = cv2.VideoWriter_fourcc(*'XVID') vid = cv2.VideoWriter('output.avi', fourcc, 6, (640,480)) 

and inside the loop you can add:

 vid.write(frame) #the edited frame or the original img_np as you please 

UPDATE
the end result looks something like this (if you want to achieve the stream of frames that you have. Storing as a video is just a demonstration of the use of opencv on the screen):

 from PIL import ImageGrab import numpy as np import cv2 while(True): img = ImageGrab.grab(bbox=(100,10,400,780)) #bbox specifies specific region (bbox= x,y,width,height) img_np = np.array(img) frame = cv2.cvtColor(img_np, cv2.COLOR_BGR2GRAY) cv2.imshow("test", frame) cv2.waitKey(0) cv2.destroyAllWindows() 

Hope that helps

Answer 3

I tried all of the above, but he did not give me real-time screen updates. You can try this. This code is tested and works successfully, and also gives good fps output. You can also judge this by each necessary cycle.

 import numpy as np import cv2 from PIL import ImageGrab as ig import time last_time = time.time() while(True): screen = ig.grab(bbox=(50,50,800,640)) print('Loop took {} seconds',format(time.time()-last_time)) cv2.imshow("test", np.array(screen)) last_time = time.time() if cv2.waitKey(25) & 0xFF == ord('q'): cv2.destroyAllWindows() break 

Answer 4

You can try this =>

 import mss import numpy with mss.mss() as sct: monitor = {'top': 40, 'left': 0, 'width': 800, 'height': 640} img = numpy.array(sct.grab(monitor)) print(img) 

Answer 5

Based on this post and other posts, I did something like this. This is a screen shot and recording to a video file without saving img.

 import cv2 import numpy as np import os import pyautogui output = "video.avi" img = pyautogui.screenshot() img = cv2.cvtColor(np.array(img), cv2.COLOR_RGB2BGR) #get info from img height, width, channels = img.shape # Define the codec and create VideoWriter object fourcc = cv2.VideoWriter_fourcc(*'mp4v') out = cv2.VideoWriter(output, fourcc, 20.0, (width, height)) while(True): try: img = pyautogui.screenshot() image = cv2.cvtColor(np.array(image), cv2.COLOR_RGB2BGR) out.write(img) StopIteration(0.5) except KeyboardInterrupt: break out.release() cv2.destroyAllWindows() 

Answer 6

I tried ImageGrab from PIL and it gave me 20fps, which is fine, but using win32 libraries gave me + 40fps, which is amazing!

I used this code from Frannecklp, but it did not work fine, so I needed to change it:

-Firstly pip install pywin32 in case of using libraries

-import libraries like this:

 import cv2 import numpy as np from win32 import win32gui from pythonwin import win32ui from win32.lib import win32con from win32 import win32api 

to get a simple screen image do:

 from grab_screen import grab_screen import cv2 img = grab_screen() cv2.imshow('frame',img) 

and to get frames:

 while(True): #frame = grab_screen((0,0,100,100)) frame = grab_screen() cv2.imshow('frame',frame) if cv2.waitKey(1) & 0xFF == ord('q') or x>150: break 

Answer 7

With all of the above solutions, I could not get a suitable frame rate until I changed my code as follows:

 import numpy as np import cv2 from mss import mss from PIL import Image bbox = {'top': 100, 'left': 0, 'width': 400, 'height': 300} sct = mss() while 1: sct_img = sct.grab(bbox) cv2.imshow('screen', np.array(sct_img)) if cv2.waitKey(1) & 0xFF == ord('q'): cv2.destroyAllWindows() break 

With this solution, I easily get 20+ frames per second.

For reference, check out this link: https://python-mss.readthedocs.io/examples.html

Answer 8

You can try this. This code works for me. I tested Linux

 import numpy as np import cv2 from mss import mss from PIL import Image sct = mss() while 1: w, h = 800, 640 monitor = {'top': 0, 'left': 0, 'width': w, 'height': h} img = Image.frombytes('RGB', (w,h), sct.grab(monitor).rgb) cv2.imshow('test', np.array(img)) if cv2.waitKey(25) & 0xFF == ord('q'): cv2.destroyAllWindows() break 

make sure the following package is installed

Pillow, OpenCV-Python, NumPy, MCC