The easiest (legal) way to programmatically get a google search results counter?

Question

I want to get an estimated result for specific Google search engine searches (across the network) using Java code.

I only need to make very few requests per day, so at first the Google Web Search API , although outdated, seemed pretty good (see for example How you can search the Google APIs ). But as it turned out, the numbers returned by this API are very different from the numbers returned by www.google.com (see for example http://code.google.com/p/google-ajax-apis/issues/detail?id= 32 ). Therefore, these figures are useless to me.

I also tried Google Custom Search , which shows the same problem.

Do you think this is the easiest solution for my task?

Answer 1

/**** @author RAJESH Kharche */ //open Netbeans //Choose Java->prject //name it GoogleSearchAPP package googlesearchapp; import java.io.*; import java.net.*; import java.util.*; import java.util.logging.Level; import java.util.logging.Logger; public class GoogleSearchAPP { public static void main(String[] args) { try { // TODO code application logic here final int Result; Scanner s1=new Scanner(System.in); String Str; System.out.println("Enter Query to search: ");//get the query to search Str=s1.next(); Result=getResultsCount(Str); System.out.println("Results:"+ Result); } catch (IOException ex) { Logger.getLogger(GoogleSearchAPP.class.getName()).log(Level.SEVERE, null, ex); } } private static int getResultsCount(final String query) throws IOException { final URL url; url = new URL("https://www.google.com/search?q=" + URLEncoder.encode(query, "UTF-8")); final URLConnection connection = url.openConnection(); connection.setConnectTimeout(60000); connection.setReadTimeout(60000); connection.addRequestProperty("User-Agent", "Google Chrome/36");//put the browser name/version final Scanner reader = new Scanner(connection.getInputStream(), "UTF-8"); //scanning a buffer from object returned by http request while(reader.hasNextLine()){ //for each line in buffer final String line = reader.nextLine(); if(!line.contains("\"resultStats\">"))//line by line scanning for "resultstats" field because we want to extract number after it continue; try{ return Integer.parseInt(line.split("\"resultStats\">")[1].split("<")[0].replaceAll("[^\\d]", ""));//finally extract the number convert from string to integer }finally{ reader.close(); } } reader.close(); return 0; } } 

Answer 2

Something you can do is do an actual Google search programmatically for a start. The easiest way to do this is to access the URL https://www.google.com/search?q=QUERY_HERE , and then you want to clear the result from this page.

Here is a brief example of how to do this:

  private static int getResultsCount(final String query) throws IOException { final URL url = new URL("https://www.google.com/search?q=" + URLEncoder.encode(query, "UTF-8")); final URLConnection connection = url.openConnection(); connection.setConnectTimeout(60000); connection.setReadTimeout(60000); connection.addRequestProperty("User-Agent", "Mozilla/5.0"); final Scanner reader = new Scanner(connection.getInputStream(), "UTF-8"); while(reader.hasNextLine()){ final String line = reader.nextLine(); if(!line.contains("<div id=\"resultStats\">")) continue; try{ return Integer.parseInt(line.split("<div id=\"resultStats\">")[1].split("<")[0].replaceAll("[^\\d]", "")); }finally{ reader.close(); } } reader.close(); return 0; } 

To use, you should do something like:

 final int count = getResultsCount("horses"); System.out.println("Estimated number of results for horses: " + count);