Perl module to get all pages of a website?

Question

Is there a module that can give me links to all the pages that are on the website?

Why do I need this: I want to scan some sites and search for tags in them, searching only on the main page is not enough.

Thank,

Answer 1

HTML :: SimpleLinkExtor is a bit simpler than HTML :: LinkExtor . You can check out my half try in my web tool , which has some code that you probably need.

Answer 2

Perl WWW:: Mechanize, , . , , follow_link() get() .

Answer 3

HTML::LinkExtor.

Answer 4

- HTML:: TreeBuilder HTML . , , , , . HTML:: Element, .

:

use HTML::TreeBuilder;
use LWP::Simple;

my $url  = 'http://www.example.com/';
my $html = HTML::TreeBuilder->new_from_content(get($url));

my @links = $html->look_down('_tag'   => 'a');

, LWP:: Simple HTML:: TreeBuilder Ubuntu.