Getting table text fast in Selenium

Question

I am trying to parse several columns in a table in a dictionary using Selenium, but what seems to me to be slow. I am using python, Selenium 2.0 and webdriver.Chrome ()

table = self.driver.find_element_by_id("thetable") # now get all the TR elements from the table all_rows = table.find_elements_by_tag_name("tr") # and iterate over them, getting the cells for row in all_rows: cells = row.find_elements_by_tag_name("td") # slowwwwwwwwwwwwww dict_value = {'0th': cells[0].text, '1st': cells[1].text, '2nd': cells[2].text, '3rd': cells[3].text, '6th': cells[6].text, '7th': cells[7].text, '10th': cells[10].text} 

It seems that the problem is getting the "text" attribute for each td element. Is there a faster way?

Answer 1

Alternative option.

If later (after the cycle) you do not need the interactivity that selenium gives you, you can transfer the current HTML source code of the lxml.html page, which is known for its speed. Example:

 import lxml.html root = lxml.html.fromstring(driver.page_source) for row in root.xpath('.//table[@id="thetable"]//tr'): cells = row.xpath('.//td/text()') dict_value = {'0th': cells[0], '1st': cells[1], '2nd': cells[2], '3rd': cells[3], '6th': cells[6], '7th': cells[7], '10th': cells[10]}