Jul-10-2018, 08:35 AM
(This post was last modified: Jul-10-2018, 08:35 AM by Prince_Bhatia.)
i am trying to scrape few pages via reading them through csv. This is one of the example web page http://up-rera.in/AgentDetails.aspx?Id=10868
i am trying to scrape name and mobile values.
but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000
below is my code:
i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values
Please help
i am trying to scrape name and mobile values.
but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000
below is my code:
import requests from bs4 import BeautifulSoup import time import os import csv file = {} final_data = [] def read_file(): global file f = open("Data.csv") for line in f.readlines(): data = line.split(",") file[data[0]]= data[1] def writedata(alldata1, filename): with open("./"+filename, "w")as csvfile: csvfile = csv.writer(csvfile, delimiter=",") csvfile.writerow("") for i in range(0, len(alldata1)): csvfile.writerow(alldata1[i]) def parse_data(): global file global final_data for data_obj in file.keys(): link = file[data_obj] #print(link) data = getbyget(link, {}) soup = BeautifulSoup(data, "html.parser") get_details = soup.find_all("div", {"class":"col-sm-12"})[0] #print(get_details) mobile = get_details.find_all("span", {"id":"ctl00_ContentPlaceHolder1_lblAname"}) for i in mobile: datas = i.text print(datas) return final_data def getbyget(url, values): res = requests.get(url, data=values) data = res.text return data def main(): read_file() datas = parse_data() writedata(datas, "UP_DATA.csv") main()i am trying to scrape this webpage from last 24 hours and same output i received. just found out __EVENTTARGET , __EVENTARGUMENT, __VIEWSTATE are all hidden, it there something related this with
i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values
Please help