Parsing a website with Jsoup that loads dynamically when custom scrolls

Question

I am working on a project that displays song names and a link to this song by analyzing a website using Jsoup. The only problem is that I can only get the first 10 elements that I want from this website, because when you scroll down, the website generates more elements. The specific website I'm trying to analyze is the TrappedIO music site. You will notice when browsing the website that when you scroll down, new song names and images will appear. When I use the check element in chrome, I see that when scrolling, it generates more elements that I'm trying to parse.

CSS path to what I parse: #content> div.container> div> div: nth-child (element index)

The problem is that I get this site using Jsoup using this method,

Document doc = Jsoup.connect(url).get(); 

HTML returns from Jsoup: Pastebin

Only the first 10 elements that I want to parse are returned, along with all the other HTML. To be more specific, I understand using Jsoup in AsyncTask, and then populate the ListView with the analyzed data.

Any ideas? Any suggestions on how to download all at once? Any answer is much appreciated, thanks.

Answer 1

Its quite simple to get the next set of 10, just click

 http://trapped.io/?page=2 

To summarize, simply enter the correct page number in the query parameter page=PAGE_NUMBER , you can get a set of 10 elements on this page.

Edit:

Just note that there may be a problem in scraping sites. I hope you double checked its legal approval to do a scraping on your site.