AJAX process request in Htmlunit

Question

I have a program written to clear the source code from a web page after clicking a button. I cannot clear the correct page because I believe the AJAX request is being sent, and I do not expect this answer to occur. My code is:

public class Htmlunitscraper { private static String s = "http://cpdocket.cp.cuyahogacounty.us/SheriffSearch/results.aspx?q=searchType%3dSaleDate%26searchString%3d10%2f21%2f2013%26foreclosureType%3d%27NONT%27%2c+%27PAR%27%2c+%27COMM%27%2c+%27TXLN%27"; public static String scrapeWebsite() throws IOException { java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF); System.setProperty("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog"); final WebClient webClient = new WebClient(); final HtmlPage page = webClient.getPage(s); final HtmlForm form = page.getForms().get(2); final HtmlSubmitInput button = form.getInputByValue(">"); final HtmlPage page2 = button.click(); String originalHtml = page2.refresh().getWebResponse().getContentAsString(); return originalHtml; } } 

After linking to the link, I believe that I fixed this, I was able to implement the method "webClient.waitForBackgroundJavaScript (10000)". The only problem is that I don’t understand how to do this, because every time I click the button, I create an HtmlPage object, not a WebClient object. How can I enable this method to fix the problem?

Answer 1

For me it helped to use htmlunit 2.15 with NicelyResynchronizingAjaxController as well

 webClient.getOptions().setThrowExceptionOnScriptError(false); 

My complete setup

  WebClient webClient = new WebClient(BrowserVersion.FIREFOX_24); webClient.getOptions().setJavaScriptEnabled(true); webClient.getOptions().setThrowExceptionOnScriptError(false); webClient.getOptions().setCssEnabled(false); webClient.setAjaxController(new NicelyResynchronizingAjaxController()); 

Answer 2

I would try a setup solution

 webClient.setAjaxController(new NicelyResynchronizingAjaxController()); 

this will cause the synchronization of all ajax calls.

Alternatively, did you try in your decision to call "webClient.waitForBackgroundJavaScript (10000)" after it received the page?

Something like that:

 final HtmlPage page2 = button.click(); webClient.waitForBackgroundJavaScript(10000) String originalHtml = page2.asXml(); return originalHtml; 

Also use htmlunit 2.13