Java find screen image

Question

Can you guys give me hints on how to find the image on the screen. I mean, a simple combination of pixels. For exmaple, it finds the coordinates of a white square of 30x30 pixels.

The Java robot class allows me to find the color of a specific pixel. But I need the other way around, I want my program to scan my screen, and then tell me the coordinates of this small image. Well, I could go through all the pixels with a robot, but it should be faster than that. Much faster.

Any suggestions?

Answer 1

Well, I could go through all the pixels with a robot, but it should be faster than that. Much faster.

I am afraid that this is exactly what you will need to do.

If all the pixels should be white, you can first perform 30-pixel wide steps, and if you find a white pixel, take, say, 5 pixel steps, and then, if these pixels are also white, look at the remaining pixels in the square.

Something like that:

. . . . . . . .......... . . . ...... . . . . . . . . . . . .......... . .......... .......... .......... .......... . . . .......... 

Answer 2

In fact, there is a lot of simplified or more reliable solution. You can implement Sikuli libraries inside your Java application to display and interact with image elements on your screen. It was intended to automate user interface testing, but I think it can easily satisfy your needs.

Application example ( source ):

 import java.net.MalformedURLException; import java.net.URL; import org.sikuli.api.*; import org.sikuli.api.robot.Mouse; import org.sikuli.api.robot.desktop.DesktopMouse; import org.sikuli.api.visual.Canvas; import org.sikuli.api.visual.DesktopCanvas; import static org.sikuli.api.API.*; public class HelloWorldExample { public static void main(String[] args) throws MalformedURLException { // Open the main page of Google Code in the default web browser browse(new URL("http://code.google.com")); // Create a screen region object that corresponds to the default monitor in full screen ScreenRegion s = new DesktopScreenRegion(); // Specify an image as the target to find on the screen URL imageURL = new URL("http://code.google.com/images/code_logo.gif"); Target imageTarget = new ImageTarget(imageURL); // Wait for the target to become visible on the screen for at most 5 seconds // Once the target is visible, it returns a screen region object corresponding // to the region occupied by this target ScreenRegion r = s.wait(imageTarget,5000); // Display "Hello World" next to the found target for 3 seconds Canvas canvas = new DesktopCanvas(); canvas.addLabel(r, "Hello World").display(3); // Click the center of the found target Mouse mouse = new DesktopMouse(); mouse.click(r.getCenter()); } } 

Also see How to use Sikuli inside your Java programs for customization.