How to interact with a website using HttpURLConnection?

Question

Here is the story: there is a site with fields of some form. After filling in the fields, the user can click on the search. By clicking the search button, you will see a list of links and will be shown under the form. If the number of links is more than 10 , the user can click on the next button, which is the javascript button .

Part of my Java program connects to this site using HttpUrlConnection and submits form data through an established connection. Getting the first 10 links is not a problem ; I mean linking to the website, sending field data and getting the first 10 links is not a problem. My problem is how to get the following 10 links.

Thanks for the help.

Answer 1

Take a look at HtmlUnit . This structure behaves like a browser and allows you to programmatically navigate between pages as you wish. It also runs Javascript embedded in pages.