Filling out web form data using Python built-in modules

Question

Okay, so I used mechanization, queries, a wonderful soup and even selenium to my enterprise to do something similar, and I came to the conclusion that urllib and other default modules are the best way to go. The only problem is that I can’t understand how to use it at all. So can someone please show me some good places to find out specifically about this? I also have a better understanding of examples, so if someone converts this to what I ask, that would be great (also including the submit button LOL)

from selenium import webdriver driver = webdriver.Firefox() driver.get("http://www.jonessoda.com/contests/back2school") element = driver.find_element_by_name("fname") element.send_keys("Ben") 

Answer 1

You need Selenium. It simulates the interaction of a graphical interface in a browser. When you do things like entering data on a competition form, this will be the method that will be the least detectable.

Selenium Note: This is not a language library. For each language, there are special bindings for the client. Most examples and best practices are actually written in Java.

Good Selenium-python resource

Here is your working example. Including submit button.

 from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC i = 2 # do it 2 times while i > 0: driver = webdriver.Firefox() driver.get("http://www.jonessoda.com/contests/back2school") def find_by_xpath(locator): element = WebDriverWait(driver, 10).until( EC.presence_of_element_located((By.XPATH, locator)) ) return element class FormPage(object): def fill_form(self, data): find_by_xpath('//input[@name = "fname"]').send_keys(data['fname']) find_by_xpath('//input[@name = "lname"]').send_keys(data['lname']) find_by_xpath('//input[@name = "email"]').send_keys(data['email']) find_by_xpath('//select[@name = "birthday_month"]').send_keys(data['month']) find_by_xpath('//select[@name = "birthday_day"]').send_keys(data['day']) find_by_xpath('//select[@name = "birthday_year"]').send_keys(data['year']) return self # makes it so you can call .submit() after calling this function def submit(self): find_by_xpath('//input[@value = "Submit"]').click() data = { 'fname': 'Sheep', 'lname': 'Test', 'email': ' [e-mail] ', 'month': 'October', 'day': '29', 'year': '1920' } FormPage().fill_form(data).submit() driver.quit() # closes the webbrowser i = i - 1