How to read text from an image (captcha) using Selenium WebDriver with Java

Question

I have a registration webpage, but is displayed in the last picture ..

I can not read the text from the image. I am going to mention the code and output.

@Test public void loginTest() throws InterruptedException { System.out.println("Testing"); driver.get("https://customer.onlinelic.in/ForgotPwd.htm"); WebElement element = driver.findElement(By.xpath("//*[@id='forgotPassword']/table/tbody/tr[5]/td[3]/img")); System.out.println(" get the instance "); String elementTest = element.getAttribute("src"); System.out.println("Element : " + elementTest); } 

Output: error

An exception in the stream "main" org.openqa.selenium.NoSuchElementException: Could not find an element: {"Method": "XPath", "selector": "// [@ ID = 'forgotPassword'] / table / TBODY / tr [ 5] / td [3] / IMG "} Duration of the command or timeout: 60.02 seconds. For documentation of this error, please visit: http://seleniumhq.org/exceptions/no_such_element.html Assembly Information: Version: '2.35.0', Version: '8df0c6b', Time: '2013-08-12 15 : 43: 19 'System information: os.name: "Windows 7", os.arch:' amd64 ', os.version:' 6.1 ', java.version:' 1.6.0_26 'Session ID: 5f5b2e1a-56a4-49ad -8fd3-2870747a7768 Driver Information: org.openqa.selenium.firefox.FirefoxDriver Capabilities [{platform = XP, acceptSslCerts = true, javascriptEnabled = true, browserName = firefox, rotatable = false, locationContextEnabled = true, version = 23.0.1 cssSelectorsEnabled = true, databaseEnabled = true, handleAlerts = true, browserConnectionEnabled = true, nativeEvents = true, webStorageEnabled = true, applicationCacheEnabled = true, takes Screenshot = true}] in sun.reflect.NativeConstructorAccessorImpl. newInstance0 (native method) in sun.reflect.NativeConstructorAccessorImpl.newInstance (NativeConstructorAccessorImpl.java:39) in sun.reflect.DelegatingConstructorAccessorImpl.newInstance (DelegatingConstructorAccessorImpl.java:27) in java.l.angctorn.constructor.structor..unstruang.neworon.neworon.neworon.neworon.neworn.nonector 513) in org.openqa.selenium.remote.ErrorHandler.createThrowable (ErrorHandler.java:191) in org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed (ErrorHandler.java:145) in org.openqareote.reemote.reemote.reemote.reemote.reote .execute (RemoteWebDriver.java∗54) in org.openqa.selenium.remote.RemoteWebDriver.findElement (RemoteWebDriver.java:307) in org.openqa.selenium.remote.RemoteWebDriver.findElementByXPath (RemoteWebDver. openqa.selenium.By $ ByXPath.findElement (By.java{44) at org.openqa.selenium.remote.RemoteWebDriver.findElement (RemoteWebDriver.java:299) in seleniumtest.CaptchaTest.loginTest (CaptchaTest.java:41) .CaptchaTest.main (Cap tchaTest.java:59) Called: org.openqa.selenium.remote.ErrorHandler $ UnknownServerException: Unable to find element: {"Method": "XPath", "selector": "// [@ ID = 'forgotPassword'] / table / TBODY / mp [5] / etc [3] / IMG "} Assembly information: version:" 2.35.0 ", version:" 8df0c6b ", time:" 2013-08-12 15:43:19 'System information : os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.6.0_26' Driver information: driver.version: unknown in .FirefoxDriver.prototype. findElementInternal_ (file: /// C: / Users / lukup / AppData / Local / Temp / anonymous4043037924964932185webdriver-profile / extensions / [e-mail] /components/driver_component.js: 8880) on .fxdriver.Timer.prototype.setToutout / set .notify (file: /// C: / Users / lukup / AppData / Local / Temp / anonymous4043037924964932185webdriver-profile / extensions / [e-mail] / components / driver_compone nt.js: 396)

Answer 1

To spell out the previous answers in detail, CAPTCHA stands for Turing's Fully Automated Public Test to Talk About Computers and People. Thus, if the “machine” can solve this problem, it really does not do the job.

To solve this problem, you can do something - use the API of external services, such as http://www.deathbycaptcha.com . You implement your API by passing them a CAPTCHA and returning the text. The average solution time that I observed is about 10-15 seconds.

Implementation example (taken from here )

 import com.DeathByCaptcha.AccessDeniedException; import com.DeathByCaptcha.Captcha; import com.DeathByCaptcha.Client; import com.DeathByCaptcha.SocketClient; import com.DeathByCaptcha.HttpClient; /* Put your DeathByCaptcha account username and password here. Use HttpClient for HTTP API. */ Client client = (Client)new SocketClient(username, password); try { double balance = client.getBalance(); /* Put your CAPTCHA file name, or file object, or arbitrary input stream, or an array of bytes, and optional solving timeout (in seconds) here: */ Captcha captcha = client.decode(captchaFileName, timeout); if (null != captcha) { /* The CAPTCHA was solved; captcha.id property holds its numeric ID, and captcha.text holds its text. */ System.out.println("CAPTCHA " + captcha.id + " solved: " + captcha.text); if (/* check if the CAPTCHA was incorrectly solved */) { client.report(captcha); } } } catch (AccessDeniedException e) { /* Access to DBC API denied, check your credentials and/or balance */ } 

Answer 2

Two problems.

• You have the wrong xpath, so you get a NoSuchElement exception.

• Even you had the correct xpath, you would not be able to extract the text as this will defeat the point if CAPTCHA

Answer 3

I have a solution that will work on a specific website. You can take a snapshot of the entire page and get a captcha image. Then divide the entire width of the captcha image by the total number of characters (usually it’s usually constant in full). Now we have individual captcha image characters. Collect all possible captcha characters by reloading the page.

After you have all the possible characters, after which you will get some captcha image, you can compare its characters with the images that we have and decide which letter or number it has.

Stages:

• Collect the captcha image and separate it into separate characters.

   private static BufferedImage cropImage(File filePath, int x, int y, int w, int h) { try { BufferedImage originalImgage = ImageIO.read(filePath); BufferedImage subImgage = originalImgage.getSubimage(x, y, w, h); return subImgage; } catch (IOException e) { e.printStackTrace(); return null; } } 

  2. Save all possible images in a folder

  3. Now read each image of the captcha symbol and compare it with all the other images in the specified folder. You can compare two images using pixel values. Public static float getDiff (File f1, File f2, width int, int height) throws IOException {BufferedImage bi1 = null; BufferedImage bi2 = null; bi1 = new BufferedImage (width, height, BufferedImage.TYPE_INT_ARGB); bi2 = new BufferedImage (width, height, BufferedImage.TYPE_INT_ARGB);

       bi1 = ImageIO.read(f1); bi2 = ImageIO.read(f2); float diff = 0; for (int i = 0; i < width; i++) { for (int j = 0; j < height; j++) { int rgb1 = bi1.getRGB(i, j); int rgb2 = bi2.getRGB(i, j); int b1 = rgb1 & 0xff; int g1 = (rgb1 & 0xff00) >> 8; int r1 = (rgb1 & 0xff0000) >> 16; int b2 = rgb2 & 0xff; int g2 = (rgb2 & 0xff00) >> 8; int r2 = (rgb2 & 0xff0000) >> 16; diff += Math.abs(b1 - b2); diff += Math.abs(g1 - g2); diff += Math.abs(r1 - r2); } } return diff; } 

Whatever image is with a lower difference value, which is an actual match. Add its name to the string. After reading all the images, captcha 1's return line: https://i.stack.imgur.com/FYPhd.png

In the above image, the image indicates a number or symbol.

This only works for a simple captcha such as [ 1

Answer 4

And here is an example code for reading text from the image above:

 import java.awt.Image; import java.awt.image.BufferedImage; import java.awt.image.RenderedImage; import java.io.File; import java.io.IOException; import java.net.URL; import javax.imageio.ImageIO; import org.openqa.selenium.WebDriver; import org.openqa.selenium.firefox.FirefoxDriver; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; import com.asprise.util.ocr.OCR; public class ExtractImage { WebDriver driver; @BeforeTest public void setUpDriver() { driver = new FirefoxDriver(); } @Test public void start() throws IOException{ /*Navigate to http://www.mythoughts.co.in/2013/10/extract-and-verify-text-from-image.html page * and get the image source attribute * */ driver.get("http://www.mythoughts.co.in/2013/10/extract-and-verify-text-from-image.html"); String imageUrl=driver.findElement(By.xpath("//*[@id='post-body-5614451749129773593']/div[1]/div[1]/div/a/img")).getAttribute("src"); System.out.println("Image source path : \n"+ imageUrl); URL url = new URL(imageUrl); Image image = ImageIO.read(url); String s = new OCR().recognizeCharacters((RenderedImage) image); System.out.println("Text From Image : \n"+ s); System.out.println("Length of total text : \n"+ s.length()); driver.quit(); /* Use below code If you want to read image location from your hard disk * BufferedImage image = ImageIO.read(new File("Image location")); String imageText = new OCR().recognizeCharacters((RenderedImage) image); System.out.println("Text From Image : \n"+ imageText); System.out.println("Length of total text : \n"+ imageText.length()); */ } } 

The program description is displayed here:

Image Source Path: http://2.bp.blogspot.com/-42SgMHAeF8U/Uk8QlYCoy-I/AAAAAAAAADSA/TTAVAAgDhio/s1600/love.jpg

Never use M2 to use O, ne Who you like Never say what is busy, Who you need Never fool The One Who ReaZZy trusts you, Never delete one Who calls you.

Length of entire text: 175

Answer 5

The forgotten password form is in the iframe. That is why selenium does not find the element. You need to switch to the iframe first while holding the form, and then launch your element. Your xpath is correct.

Use driver.switchTo().frame(arg0) to switch to frame. See javadoc here

To get the captcha text, I didn’t understand what you mean by “test and comparison storage”. Ideally, you will not be able to read the text from the captcha (as others said). One alternative approach I've seen is to save the captcha value as alt text in the development environment and QA. So you can read it and enter in the text box. When the code goes to production or any external environment, this alt text can be deleted.

Answer 6

The whole purpose of CAPTCHA is to prevent automation from the user interface! You can use the internal APIs to validate the action.

Answer 7

For a simple captcha, you can call the free ocr.space online OCR service . While this is only an image with slightly distorted text, it works fine. You can also use Google Cloud Vision OCR , and it works equally well, but it's a little harder to set up.

Answer 8

Cannot read CAPTCHA. If you can read CAPTCHA, it makes no sense to use CAPTCHA.