How can I activate each element and analyze their information?

Question

I ran into another type of problem when clearing a webpage using python. When the image is clicked, new information about its "flavor" appears. My goal is to analyze all the tastes associated with each image. My script can analyze the tastes of the current active image, but it breaks after clicking on a new image. A slight twitch in my loop will lead me in the right direction.

I tried:

from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC driver = webdriver.Chrome() driver.get("https://www.optigura.com/uk/product/gold-standard-100-whey/") wait = WebDriverWait(driver, 10) while True: items = wait.until(EC.presence_of_element_located((By.XPATH, "//div[@class='colright']//ul[@class='opt2']//label"))) for item in items.find_elements_by_xpath("//div[@class='colright']//ul[@class='opt2']//label"): print(item.text) try: links = driver.find_elements_by_xpath("//span[@class='img']/img") for link in links: link.click() except: break driver.quit() 

The image below may make it clear that I could not:

Answer 1

I set up the code to correctly click the links and check if the current text of the element matches the active text of the element. If they match, you can safely continue parsing without worrying about repeating the same thing over and over. Here you are:

 from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC driver = webdriver.Chrome() driver.get("https://www.optigura.com/uk/product/gold-standard-100-whey/") wait = WebDriverWait(driver, 10) links = driver.find_elements_by_xpath("//span[@class='img']/img") for idx, link in enumerate(links): while True: try: link.click() while driver.find_elements_by_xpath("//span[@class='size']")[idx].text != driver.find_elements_by_xpath("//div[@class='colright']//li[@class='active']//span")[1].text: link.click() print driver.find_elements_by_xpath("//span[@class='size']")[idx].text items = wait.until(EC.presence_of_element_located((By.XPATH, "//div[@class='colright']//ul[@class='opt2']//label"))) for item in items.find_elements_by_xpath("//div[@class='colright']//ul[@class='opt2']//label"): print(item.text) except StaleElementReferenceException: continue break driver.quit() 

Answer 2

I don't think this has much to do with Python, just a lot of Javascript and ajax.

Javascript part

 $(document).on("click", ".product-details .custom-radio input:not(.active input)", function() { var elm = $(this); var root = elm.closest(".product-details"); var option = elm.closest(".custom-radio"); var opt, opt1, opt2, ip, ipr; elm.closest("ul").find("li").removeClass("active"); elm.closest("li").addClass("active"); if (option.hasClass("options1")) { ip = root.find(".options1").data("ip"); opt = root.find(".options2").data("opt"); opt1 = root.find(".options1 li.active input").val(); opt2 = root.find(".options2 li.active input").data("opt-sel"); } else ipr = root.find(".options2 input:checked").val(); $.ajax({ type: "POST", url: "/product/ajax/details.php", data: { opt: opt, opt1: opt1, opt2: opt2, ip: ip, ipr: ipr }, 

So you can just create the parameters (in this case, using the css selector would be better than xpath), post and analyze the json results.