Why doesn't lxml find the xpath specified by the Chrome inspector?

Question

Here is my code:

from lxml import html import requests page = requests.get('https://en.wikipedia.org/wiki/Nabucco') tree = html.fromstring(page.content) title = tree.xpath('//*[@id="mw-content-text"]/table[1]/tbody/tr[1]/th/i') print(title) 

Problem: print (title) prints "[]", an empty list. I expect this to print "Nabucco". The XPath expression refers to the Copy XPath function of the Chrome Inspector.

Why is this not working? Is there a disagreement between lxml and the Chrome xpath engine? Or am I missing something? I am somewhat new to python, lxml and xpath.

Answer 1

This is because of the tbody tag. You see this in the browser because the tag is inserted by the browser. requests not a browser and just loads the page source as it is:

Replace:

 //*[@id="mw-content-text"]/table[1]/tbody/tr[1]/th/i 

with:

 //*[@id="mw-content-text"]/table[1]/tr[1]/th/i