How to programmatically log into a site in Python

Question

I searched all over the Internet, looking at a lot of examples and tried everything that I found, but none of them work for me, so please do not think that this is a duplicate. I need help with my specific case.

I am trying to enter the website using Python (in this case I am trying to use v2.7, but I am not opposed to using a newer version, I just could find the most complete information about 2.7).

I need to fill out a short form consisting simply of a username and password. The web page form that I need to fill out and log in is as follows (this is random, I know):

<form method="post" action="login.aspx?ReturnUrl=..%2fwebclient%2fstorepages%2fviewshifts.aspx" id="Form1">
<div class="aspNetHidden">
<input type="hidden" name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwUKMTU4MTgwOTM1NWRkBffWXYjjifsi875vSMg9OVkhxOQYYstGTNcN9/PFb+M=" />
</div>

<div class="aspNetHidden">

    <input type="hidden" name="__EVENTVALIDATION" id="__EVENTVALIDATION" value="/wEdAAVrmuRkG3j6RStt7rezNSLKVK7BrRAtEiqu9nGFEI+jB3Y2+Mc6SrnAqio3oCKbxYY85pbWlDO2hADfoPXD/5td+Ot37oCEEXP3EjBFcbJhKJGott7i4PNQkjYd3HFozLgRvbhbY2j+lPBkCGQJXOEe" />
</div>
            <div><span></span>
                <table style="BORDER-COLLAPSE: collapse" borderColor="#000000" cellSpacing="0" cellPadding="0"
                    width="600" align="center" border="1">
                    <tr>
                        <td>
                            <table cellSpacing="0" cellPadding="0" width="100%" align="center" border="0">
                                <tr>
                                    <td width="76%"><span id="centercontentTitle"></span>
                                        <H1 align="center"><br>
                                            <span>
                                                <IMG height="52" src="../images/logo-GMR.jpg" width="260"></span><span><br>
                                            </span></H1>
                                        <div id="centercontentbody">
                                            <div align="center">
                                                <TABLE width="350">
                                                    <TR>
                                                        <TD class="style7">Username:</TD>
                                                        <TD>
                                                            <div align="right"><input name="txtUsername" type="text" id="txtUsername" style="width:250px;" /></div>
                                                        </TD>
                                                    </TR>
                                                    <TR>
                                                        <TD class="style7">Password:</TD>
                                                        <TD>
                                                            <div align="right"><input name="txtPassword" type="password" id="txtPassword" style="width:250px;" /></div>
                                                        </TD>
                                                    </TR>
                                                    <TR>
                                                        <TD></TD>
                                                        <TD align="right"><input type="submit" name="btnSubmit" value="Submit" id="btnSubmit" /><input type="submit" name="btnCancel" value="Cancel" id="btnCancel" /></TD>
                                                    </TR>
                                                    <TR>
                                                        <TD colspan="2" align="center"></TD>
                                                    </TR>
                                                </TABLE>
                                            </div>
                                        </div>
                                    </td>
                                    <td>
                                        <div align="center" style='height:250px'></div>
                                    </td>
                                </tr>
                            </table>
                        </td>
                    </tr>
                </table>
                <br>
                <br>
                <p>&nbsp;</p>
        </form>

From an Internet search, the best Python code I found to fill out this form and enter the site is as follows:

. , /, , , .

import cookielib
import urllib
import urllib2


# Store the cookies and create an opener that will hold them
cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))

# Add our headers
opener.addheaders = [('User-agent', 'LoginTesting')]

# Install our opener (note that this changes the global opener to the one
# we just made, but you can also just call opener.open() if you want)
urllib2.install_opener(opener)

# The action/ target from the form
authentication_url = '<URL I am trying to log into>'

# Input parameters we are going to send
payload = {
  '__EVENTVALIDATION': '/wEdAAVrmuRkG3j6RStt7rezNSLKVK7BrRAtEiqu9nGFEI+jB3Y2+Mc6SrnAqio3oCKbxYY85pbWlDO2hADfoPXD/5td+Ot37oCEEXP3EjBFcbJhKJGott7i4PNQkjYd3HFozLgRvbhbY2j+lPBkCGQJXOEe"',
  'txtUsername': '<USERNAME>',
  'txtPassword': '<PASSWORD>',
  }

# Use urllib to encode the payload
data = urllib.urlencode(payload)

# Build our Request object (supplying 'data' makes it a POST)
req = urllib2.Request(authentication_url, data)

# Make the request and read the response
resp = urllib2.urlopen(req)
contents = resp.read()

, , , . - , , , , , . !

, :)

Answer 1

__ EVENTVALIDATION, , , python, __EVENTVALIDATION, .

- :

import requests
from bs4 import BeautifulSoup

s = requests.session()

def get_eventvalidation():
    r = s.get("http://url.to.login.page")
    bs = BeautifulSoup(r.text)

    return bs.find("input", {"name":"__EVENTVALIDATION"}).attrs['value']

authentication_url = '<URL I am trying to log into>'

payload = {
  '__EVENTVALIDATION': get_eventvalidation(),
  'txtUsername': '<USERNAME>',
  'txtPassword': '<PASSWORD>',
  }

login = s.post(authentication_url, data=payload)

print login.text

beautifulsoup4. , .

: , __VIEWSTATE POST.