Cleaning a site using the login page

Question

I am currently logging into the site from the site using the following script.

browser = webdriver.Chrome('E:/Shared Folders/Users/runnerjp/chromedriver/chromedriver.exe')
browser.get("https://www.timeform.com/horse-racing/account/sign-in?returnUrl=%2Fhorse-racing%2F") 
time.sleep(3)
username = browser.find_element_by_id("EmailAddress")
password = browser.find_element_by_id("Password")
username.send_keys("usr")
password.send_keys("pass")
login_attempt = browser.find_element_by_xpath("//input[@type='submit']")
time.sleep(3)
login_attempt.submit()

it works, but I find that using the Chrome web driver clogs my processor. Is there any alternative code that I could use, this does not mean that I need to physically load the login page?

Answer 1

, - , .
- javascript/ajax- .., - , javascript. , , "" , phantomjs. phantomjs html javascript , chrome .

"", requests. , - "" , http ( ):

import requests
requests.get('https://api.github.com/user', auth=('user', 'pass'))

CodeMonkey

- , , , , , / .

Answer 2

requests . :

import requests
requests.get('https://api.github.com/user', auth=('user', 'pass'))

: http://docs.python-requests.org/en/master/user/authentication/

Answer 3

TestCafe.

TestCafe - - ( e2e). TestCafes Node.js WebDriver.

TestCafe . DOM- TestCafe Selectors. TestCafe JavaScript - ClientFunction (. ).

TestCafe , . . .

TestCafe :

1) , Node.js( ).

2) TestCafe open cmd :

npm install -g testcafe

- . : 1) - "test.js"

import { Selector } from ‘testcafe’;

fixture `Getting Started`
    .page `http://devexpress.imtqy.com/testcafe/example`;

test(‘My first test’, async t => {
    await t
        .typeText(‘#developer-name’, ‘John Smith’)
        .click(‘#submit-button’)
        .expect(Selector(‘#article-header’).innerText).eql(‘Thank you, John Smith!‘);
});

2) (, ), cmd:

testcafe chrome test.js

3) .

TestCafe : , ( , Pi Safari iOS), (, Sauce Labs) (, Nightmare). , TestCafe Continuous Integration.

You can use the same to scrape data and save to file easily

Answer 4

CPU , PhantomJS Chrome. PhantomJS :

https://realpython.com/blog/python/headless-selenium-testing-with-python-and-phantomjs/

Answer 5

'grab':

from grab import Grab

g = Grab()
g.go('https://www.timeform.com/horse-racing/account/sign-in?returnUrl=%2Fhorse-racing%2F')
g.doc.set_input('EmailAddress','[e-mail]')
g.doc.set_input('Password','somepass')
g.doc.submit()

print g.doc.body

Answer 6

mechanize, 3,2 .

from mechanize import Browser
import time    #just to check elapsed time and check performance
started_time = time.time()

browser = Browser()
url = 'https://www.timeform.com/horse-racing/account/sign-in?returnUrl=%2Fhorse-racing%2F'
browser.open(url)
browser.select_form(nr = 0)
browser["EmailAddress"] = 'putyouremailhere'
browser["Password"] = 'p4ssw0rd'

logged = browser.submit()
redirected_url = logged.read()
print redirected_url

#you can delete this section:
elapsed_time = time.time() - started_time
print elapsed_time,' seconds'

, !:)

Answer 7

:

• (Javascript ..), (.. ghostdriver) , ().
• , , , robobrowser (py3), mechanize browserplus. , javascript , .

Answer 8

, , , , , , , http ( URL-).

urllib .

. , , , urllib. , .

Requests

:

: , 2 id pwd, , , - , , , - user_id user_pwd id pwd . " URL"

dataToSend = {'user_id':'id you want to pass', 'user_pwd':'specify pwd here'}
# Here you can specify headers and cookie, specify if required 
response = requests.post(url, data=dataToSend, headers={'content-type':'specify if required', 'user-agent':'chrome...'})

if(response.status_code == 200):
     contentReceived = response.content
     # Here you need to observe the received content, most of the time content will be in json format, so you need to decode here.
     if(contentReceived == 'Response is same that you have expected'):
          print "Successfully"
     else:
          print "Failed"
else:
     print "Failed"

, , cookie .

Answer 9

https://scrapy.org/. , .

JavaScript, scrapy-splash: https://github.com/scrapy-plugins/scrapy-splash.

" Scrapy" : https://doc.scrapy.org/en/latest/topics/request-response.html#topics-request-response-ref-request-userlogin