How can I use python request to login to this site?

Question

I stop working with Python and web scrapers. And I can’t even achieve the first step of scraping the site: login. Before trying to use mechanize or selenium , I want to use requests first. Can anybody help me?

The website I was trying to access is here .

For those who don’t have an account and want to help me, I will provide information obtained through Chrome.

First POST: POST_1.JPG . Second POST: POST_2.JPG

And here is my code for trying to login:

 import requests s = requests.Session() headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/59.0.3071.115 Safari/537.36'} payload = {'username': 'USERNAME', 'password': 'PASSWORD', 'submit.x': '21', 'submit.y': '12'} s.post('https://ssologin.cuny.edu/oam/server/auth_cred_submit', data = payload, headers = headers) target_page = s.get('https://home.cunyfirst.cuny.edu/psp/cnyepprd/EMPLOYEE/EMPL/h/?tab=DEFAULT', headers = headers) if 'login' in target_page.url: print("Failed to Login") else: print("Logged in Successfully!") 

Thanks!!!

Answer 1

You just need to reproduce the behavior of the page as it is:

 from requests import Session login_data = { 'username' : 'YOUR_USERNAME_HERE', 'password' : 'YOUR_PASSWORD_HERE', 'submit.x' : '41', 'submit.y' : '2', } headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2228.0 Safari/537.36', 'Referer': 'https://ssologin.cuny.edu/cuny.html?resource_url=https%3A%2F%2Fhome.cunyfirst.cuny.edu%252Fpsp%252Fcnyepprd%252FEMPLOYEE%252FEMPL%252Fh%252F%3Ftab%253DDEFAULT', } s = Session() s.get("https://home.cunyfirst.cuny.edu/psp/cnyepprd/EMPLOYEE/EMPL/h/?tab=DEFAULT") s.post("https://m.addthis.com/live/red_lojson/100eng.json?sh=590&ph=1512&ivh=590&dt=5366&pdt=11272&ict=&pct=1&perf=widget%7C11273%7C1445%2Clojson%7C12846%7C236%2Csh%7C12848%7C79&rndr=render_toolbox%7C13088&cmenu=null&ppd=0&ppl=0&fbe=&xmv=0&xms=0&xmlc=0&jsfw=jquery&jsfwv=jquery-1.7.1&al=men&scr=0&scv=0&apiu=1&ba=3&sid=59646cef3b3fdec3&rev=v7.15.4-wp&pub=cunywebservices&dp=ssologin.cuny.edu&fp=cuny.html%3Fresource_url%3Dhttps%253A%252F%252Fhome.cunyfirst.cuny.edu%25252Fpsp%25252Fcnyepprd%25252FEMPLOYEE%25252FEMPL%25252Fh%25252F%253Ftab%25253DDEFAULT&pfm=0&icns=") response = s.post("https://ssologin.cuny.edu/oam/server/auth_cred_submit", data=login_data, headers=headers) if 'Sign out' in response.content: print "[*] Logged in successfully!" else: print "[!] Couldn't login..."