Java Selenium: how can I get an HTML page of a webpage without preloading the page?

Question

Using Selenium WebDriver for Java, is it possible to get the HTML page of a web page with a given URL?

I know that after loading a webpage in an HTML browser, it can be obtained using WebDriver.getPageSource (). However, to improve efficiency, is it possible to get HTML code without first loading the page in the browser?

Answer 1

You can achieve this using a headless browser.

- - . , , .

: -

• , - . , , .

• -, , . . , , -.

• . , . .

- , . , , , . , . JavaScript - , , . JavaScript - . JavaScript , , JavaScript. . , HtmlUnit JavaScript Rihno, - .

• HtmlUnit
• PhantomJS
• ZombieJS
• Watir-WebDriver

Answer 2

httpRequest JAVA:

public static String executePost(String targetURL, String urlParameters) {
  HttpURLConnection connection = null;

  try {
    //Create connection
    URL url = new URL(targetURL);
    connection = (HttpURLConnection) url.openConnection();
    connection.setRequestMethod("POST");
    connection.setRequestProperty("Content-Type", 
        "application/x-www-form-urlencoded");

    connection.setRequestProperty("Content-Length", 
        Integer.toString(urlParameters.getBytes().length));
    connection.setRequestProperty("Content-Language", "en-US");  

    connection.setUseCaches(false);
    connection.setDoOutput(true);

    //Send request
    DataOutputStream wr = new DataOutputStream (
        connection.getOutputStream());
    wr.writeBytes(urlParameters);
    wr.close();

    //Get Response  
    InputStream is = connection.getInputStream();
    BufferedReader rd = new BufferedReader(new InputStreamReader(is));
    StringBuilder response = new StringBuilder(); 

    String line;
    while ((line = rd.readLine()) != null) {
      response.append(line);
      response.append('\r');
    }
    rd.close();
    return response.toString();
  } catch (Exception e) {
    e.printStackTrace();
    return null;
  } finally {
    if (connection != null) {
      connection.disconnect();
    }
  }
}