How do I get Selenium to not wait for a page to load, which has a slow script?

Selenium driver.get (url)Wait for the page to load. But the scraping page is trying to load some dead JS script. Therefore, my Python script is waiting for it and does not work for several minutes. This problem can be on all pages of the site.

from selenium import webdriver

driver = webdriver.Chrome()
driver.get('https://www.cortinadecor.com/productos/17/estores-enrollables-screen/estores-screen-corti-3000')
# It try load: https://www.cetelem.es/eCommerceCalculadora/resources/js/eCalculadoraCetelemCombo.js 
driver.find_element_by_name('ANCHO').send_keys("100")

How to limit the timeout, block the loading of an AJAX file, or in another way?

I am also testing my script in webdriver.Chrome(), but will use PhantomJS () or possibly Firefox (). So, if any method uses changes in the browser settings, it should be universal.

+4
source share
2 answers

Selenium /url , pageLoadStrategy, normal. Selenium , pageLoadStrategy. pageLoadStrategy 3 :

  • normal ( )
  • eager ()
  • none

pageLoadStrategy:

  • Firefox:

    from selenium import webdriver
    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
    
    caps = DesiredCapabilities().FIREFOX
    caps["pageLoadStrategy"] = "normal"  #  complete
    #caps["pageLoadStrategy"] = "eager"  #  interactive
    #caps["pageLoadStrategy"] = "none"
    driver = webdriver.Firefox(desired_capabilities=caps, executable_path=r'C:\path\to\geckodriver.exe')
    driver.get("http://google.com")
    
  • Chrome:

    from selenium import webdriver
    from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
    
    caps = DesiredCapabilities().CHROME
    caps["pageLoadStrategy"] = "normal"  #  complete
    #caps["pageLoadStrategy"] = "eager"  #  interactive
    #caps["pageLoadStrategy"] = "none"
    driver = webdriver.Chrome(desired_capabilities=caps, executable_path=r'C:\path\to\chromedriver.exe')
    driver.get("http://google.com")
    
+8

Selenium Webdriver - . , WebDriver , .

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

driver = webdriver.Firefox()
driver.get("http://somedomain/url_that_delays_loading")
try:
    element = WebDriverWait(driver, 10).until(
        EC.presence_of_element_located((By.ID, "myDynamicElement"))
    )
finally:
    driver.quit()

10 , TimeoutException, 10 .

, , , , , , .

0

Source: https://habr.com/ru/post/1680162/


All Articles