How to get hidden value using beautiful soup and urllib.request

Question

I am using Python3 on Windows10X64 (Ananconda is installed). I am trying to get the value in a span element using urllib and BeautifulSoup . In Chrome, it shows 8000, but it always gives 0 letter for my code result.

Can anyone suggest a way to get the real number as shown in the Chrome web browser ?

here is my code.

from bs4 import BeautifulSoup
from urllib.request import Request, urlopen
url ='https://www.futbin.com/18/squad/100133002/sbc'
req = Request(url,headers={'User-Agent': 'Mozilla/5.0'})
page_html = urlopen(req).read()
page_soup = BeautifulSoup(page_html,'html.parser')

page_soup.findAll("div", {"class": "ps4-price"})

The result is lower.

[<div class="ps4-price">
 <img class="price-platform-img" src="https://cdn.futbin.com/design/img/logos/full_small/ps_blue.png"/>
 <div class="price-row-text" id="squad-price-ps3"><span class="psprice2">0</span></div>
 </div>]

In the span tag, it should show the same value (for example, 8000,9000) as shown in Chrome / Firefox

Answer 1

You can in this approach:

>>> url = 'https://www.futbin.com/18/squad/100133002/sbc'
>>> req = Request(url,headers={'User-Agent': 'Mozilla/5.0'})
>>> webpage = urlopen(req).read()
>>> soup = BeautifulSoup(webpage, "html.parser")
>>> required = soup.find_all("div", {"class":"pcdisplay-ps-price"})
>>> x = []
>>> for i in required:
...     x.append(i.get_text())
>>> for i in x:
...     print(i)
...

950
550
750
600
1,300
900
850
550
600
600
600

Answer 2

ps4- . REFRESH PRICE, URL-.

https://www.futbin.com/18/getPricesById

. -: https://curl.trillworks.com/

• 'https://www.futbin.com/18/squad/100133002/sbc' Chrome
• " " - "".
• REFRESH PRICE, URL- "" https://www.futbin.com/18/getPricesById '
• URL cURL
• https://curl.trillworks.com/, Python
• Since the answer is JSON, you can directly extract the price instead of bs4. print(response.json()['total_prices']['ps'])