Get a web page and save to a database?

Question

How can I get an HTML page and save it in my database in JAVA? Is there any easy way to do this?

Answer 1

Retrieving a file via http is pretty simple using the URL class

String rawHtml = IOUtils.toString(new URL("http://yahoo.com").openStream());

IOUtils is taken from org.apache.commons.io, the toString method reads the entire input stream in one line. Unfortunately, with java.net.URL you can’t control anything (cookies, header information, ..) except for the website address: - / Personally, I use this approach wherever I can, because the HttpClient API is too complicated ( too many LOCs) just to get the source code of the website.

Answer 2

.

- HttpClient

- Nutch , .