All geek questions in one place

Can lxml / request select dropdown options and then parse ajax result?

I have a site that I am trying to check, and although I can get a list of options from the drop-down list, I'm not sure how to select it? There is no submit button, so if I select it, it will load the ajax table below.

I'm just not sure lxml / requests can do this or how can this be done? I would appreciate if someone could confirm or find out a function that could do this?

edit: My site is internal and inaccessible, but here is an example site: https://www.tsx.com/listings/listing-with-us/listed-company-directory Note: if you select the field next to the search, it will pull out the results ajax (A, B, C, etc.)

+1
source share
1 answer

When setting up lxml + requests you work at a different level, at a lower level. There are no dropdown menus for requests , this is not a browser.

The general approach to your problem is as follows:

  • open the browser developer tools in your browser of choice, the "Network" tab
  • select an option from the drop-down menu
  • observe the request sent to the server on the "Network" tab
  • simulate these / these requests in your Python code

There is another “trick” that you can use to simplify the developer tools for moving to queries:

  • right-click the query in the list of queries on the Network tab.
  • click "Copy as cURL"
  • go to http://curl.trillworks.com/ and convert the cURL command to Python + Requests code
+4
source

Source: https://habr.com/ru/post/1271588/

More articles:


All Articles