Getting AJAX content using Python Selenium and PhantomJS

Question

I am writing a crawler that needs to wait for all the loaded AJAX HTML content and then extract any anchor tags from them. When I call webdriver.get (URL), I'm not sure if it really expects to add some AJAX content to the DOM. I can make some kind of implicit or explicit wait defined in the docs here . However, even after waiting, the webdriver.page_source is the same as before the wait. Anyway, can I get the AJAX content in webdriver.page_source after waiting so that I can parse asynchronously inserted content?

Answer 1

I would check for the existence of a specific element, which, as you know, is loaded asynchronously in the HTML DOM.