How can I use BeautifulSoup to search for all links on a page pointing to a specific domain?

Question

How can I use BeautifulSoup to search for all links on a page pointing to a specific domain?

Answer 1

Use SoupStrainer,

from BeautifulSoup import BeautifulSoup, SoupStrainer import re # Find all links links = SoupStrainer('a') [tag for tag in BeautifulSoup(doc, parseOnlyThese=links)] linkstodomain = SoupStrainer('a', href=re.compile('example.com/')) 

Edit: Modified example from white paper.