+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: How to enter value in gmail id and password (/thread-5996.html)
How to enter value in gmail id and password - ratanbhushan - Nov-01-2017
hi Team
I am new to Python and try to crawl gmail without using any google or gmail API.
please advise me how can i pass value to gmail id and password and then click using BeautifulSoup.
I am using IDLE python 3
THanks
Ratan
RE: How to enter value in gmail id and password - metulburr - Nov-01-2017
it would probably be easier with selenium to login and get the page content and then pass it to BS
from selenium import webdriver
from bs4 import BeautifulSoup
import time
EMAIL = ''
PASSWORD = ''
CHROMEPATH = ''
login_url = 'https://www.google.com/accounts/Login'
browser = webdriver.Chrome(CHROMEPATH)
browser.set_window_position(0,0)
browser.get(login_url)
time.sleep(2)
browser.find_element_by_id("identifierId").send_keys(EMAIL)
browser.find_element_by_id("identifierNext").click()
time.sleep(2)
browser.find_element_by_name("password").send_keys(PASSWORD)
browser.find_element_by_id("passwordNext").click()
time.sleep(2)
soup = BeautifulSoup(browser.page_source, 'html.parser')
RE: How to enter value in gmail id and password - ratanbhushan - Nov-02-2017
Hey thanks for this
I ll try selenium
tell me can we use something else other than sleep i mean is there any way to wait until the page loads
RE: How to enter value in gmail id and password - metulburr - Nov-02-2017
You can wait until an element exists
from selenium.webdriver.support.wait import WebDriverWait
element = WebDriverWait(driver, 10).until(
lambda x: x.find_element_by_id("someId"))orfrom selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by import By WebDriverWait(browser, delay).until(EC.presence_of_element_located((By.ID, 'IdOfMyElement')))And there are other variations http://www.obeythetestinggoat.com/how-to-get-selenium-to-wait-for-page-load-after-a-click.html
RE: How to enter value in gmail id and password - wavic - Nov-02-2017
Hm! I was using selenium several times and there is a delay when opening a web page. Perhaps it returns when this is done?
RE: How to enter value in gmail id and password - metulburr - Nov-02-2017
the time it takes to load a webpage can be different for each person, depending on your internet speed and computer, etc.
@OP I added the above example in for wait until elements
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from bs4 import BeautifulSoup
import time
EMAIL = ''
PASSWORD = ''
CHROMEPATH = ''
login_url = 'https://www.google.com/accounts/Login'
browser = webdriver.Chrome(CHROMEPATH)
browser.set_window_position(0,0)
wait = WebDriverWait(browser, 2)
presence = EC.presence_of_element_located
visible = EC.visibility_of_element_located
browser.get(login_url)
wait.until(presence((By.ID, 'identifierId')))
browser.find_element_by_id("identifierId").send_keys(EMAIL)
browser.find_element_by_id("identifierNext").click()
wait.until(presence((By.NAME, 'password')))
browser.find_element_by_name("password").send_keys(PASSWORD)
browser.find_element_by_id("passwordNext").click()
wait.until(visible((By.CSS_SELECTOR, '#gb > div.gb_zd.gb_5d > div.gb_yc.gb_lb.gb_xc.gb_Zd > div > div.gb_ib.gb_4c.gb_yg.gb_R.gb_pf.gb_pb > div > a > span')))
soup = BeautifulSoup(browser.page_source, 'html.parser')
print(soup.prettify())