Python selenium waiting for page to load

Question

I wrote a script that receives data from the page, but sometimes it takes time for the page to load, and so when it pulls the html into the soup object, sometimes it doesn't pull anything, since the page still needs to be completed.

I wrote the following code to wait for the page to finish.

def scrape_page(url):
     browser.get(url)    
     try:
        WebDriverWait(browser, 10).until(EC.presence_of_element_located(browser.find_element_by_id ("selection-box")))
        #Extract Source Code 
        html = browser.page_source;
        soup = BeautifulSoup(html)

Works

But I get the following error when calling a function:

TypeError: find_element() argument after * must be a sequence, not WebElement

Answer 1

I think you should use presence_of_element_locatedas follows:

element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )

as described in the manual .