get_text() returns blank value - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: get_text() returns blank value (/thread-11468.html) |
get_text() returns blank value - Prince_Bhatia - Jul-10-2018 i am trying to scrape few pages via reading them through csv. This is one of the example web page http://up-rera.in/AgentDetails.aspx?Id=10868 i am trying to scrape name and mobile values. but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000 below is my code: import requests from bs4 import BeautifulSoup import time import os import csv file = {} final_data = [] def read_file(): global file f = open("Data.csv") for line in f.readlines(): data = line.split(",") file[data[0]]= data[1] def writedata(alldata1, filename): with open("./"+filename, "w")as csvfile: csvfile = csv.writer(csvfile, delimiter=",") csvfile.writerow("") for i in range(0, len(alldata1)): csvfile.writerow(alldata1[i]) def parse_data(): global file global final_data for data_obj in file.keys(): link = file[data_obj] #print(link) data = getbyget(link, {}) soup = BeautifulSoup(data, "html.parser") get_details = soup.find_all("div", {"class":"col-sm-12"})[0] #print(get_details) mobile = get_details.find_all("span", {"id":"ctl00_ContentPlaceHolder1_lblAname"}) for i in mobile: datas = i.text print(datas) return final_data def getbyget(url, values): res = requests.get(url, data=values) data = res.text return data def main(): read_file() datas = parse_data() writedata(datas, "UP_DATA.csv") main()i am trying to scrape this webpage from last 24 hours and same output i received. just found out __EVENTTARGET , __EVENTARGUMENT, __VIEWSTATE are all hidden, it there something related this with i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values Please help RE: get_text() returns blank value - Larz60+ - Jul-10-2018 without seeing the data in 'Data.csv' (which should be named Data.html, if that what it contains, confusing otherwise) it's difficult to answer this question, as we cannot run the code. In addition, always show the entire unaltered error traceback, your synopsis is really only marginally helpful. |