Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
get_text() returns blank value
#1
i am trying to scrape few pages via reading them through csv. This is one of the example web page http://up-rera.in/AgentDetails.aspx?Id=10868

i am trying to scrape name and mobile values.

but whenever i run my code it returns BLANK. i am trying to scrape 360 REALTORS LLP and 9871898000

below is my code:

import requests
from bs4 import BeautifulSoup
import time
import os
import csv

file = {}
final_data = []

def read_file():
    global file
    f = open("Data.csv")
    for line in f.readlines():
        data = line.split(",")
        file[data[0]]= data[1]

def writedata(alldata1, filename):
    with open("./"+filename, "w")as csvfile:
        csvfile = csv.writer(csvfile, delimiter=",")
        csvfile.writerow("")
        for i in range(0, len(alldata1)):
            csvfile.writerow(alldata1[i])

def parse_data():
    global file
    global final_data
    for data_obj in file.keys():
        link = file[data_obj]
        #print(link)
        data = getbyget(link, {})
        soup = BeautifulSoup(data, "html.parser")
        get_details = soup.find_all("div", {"class":"col-sm-12"})[0]
        #print(get_details)
        mobile = get_details.find_all("span", {"id":"ctl00_ContentPlaceHolder1_lblAname"})
        for i in mobile:
            datas = i.text
            print(datas)

    return final_data

def getbyget(url, values):
    res = requests.get(url, data=values)
    data = res.text
    return data

def main():
    read_file()
    datas = parse_data()
    writedata(datas, "UP_DATA.csv")                                                      
main()
i am trying to scrape this webpage from last 24 hours and same output i received. just found out __EVENTTARGET , __EVENTARGUMENT, __VIEWSTATE are all hidden, it there something related this with

i am reading the files correctly , it is returning headings like name, Father's Name, Applicant Type but not there values

Please help
Reply
#2
without seeing the data in 'Data.csv' (which should be named Data.html, if that what it contains, confusing otherwise)
it's difficult to answer this question, as we cannot run the code. In addition, always show the entire unaltered error traceback, your synopsis is really only marginally helpful.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Flask Flash Messages Blank Leaf 2 3,588 Oct-16-2017, 10:19 PM
Last Post: Leaf

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020