Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Problem parsing website html file
#1
Hi Guys,

I am trying to parse this web link (http://insight.toba.fun/address/TVAWF6eD...sigfWK2oRB) using Python3 and requests. When we click the link it loads the details of the account. However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link. I tried putting a delay but didn't work. I'm unable to get the account the info using python. Can anyone point out a solution for this?
I tried using selenium webdriver, but I'm unable to find the search box in the html.

import requests
url='http://insight.toba.fun/address/TVAWF6eDfW9MRdLitCb7ag41sigfWK2oRB'
r = requests.get(url,verify=False)
print(r.text)
Quote
#2
Quote:However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link.
You are not saving the file.
add after line 3:
with open('yourfilename', 'w') as f:
    f.write(r.text)
(of course, replace 'yourfilername' with a valid .html file name)
Quote
#3
(May-01-2018, 11:04 AM)Larz60+ Wrote:
Quote:However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link.
You are not saving the file.
add after line 3:
with open('yourfilename', 'w') as f:
    f.write(r.text)
(of course, replace 'yourfilername' with a valid .html file name)

The OP probably meant - page is not rendered properly because of Javascript. Try https://html.python-requests.org/ which can render the page with Chromium.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  website development masoud_da 9 219 Jun-08-2019, 06:54 PM
Last Post: masoud_da
  [Flask] html error 405 SheeppOSU 0 67 Jun-08-2019, 04:42 PM
Last Post: SheeppOSU
  [split] Using beautiful soup to get html attribute value moski 6 174 Jun-03-2019, 04:24 PM
Last Post: moski
  html error 404 SheeppOSU 1 92 Jun-03-2019, 02:19 PM
Last Post: heiner55
  Dashboards with html tables fstefanov 2 167 May-08-2019, 05:53 PM
Last Post: buran
  Using beautiful soup to get html attribute value graham23s 2 186 Apr-23-2019, 09:21 PM
Last Post: graham23s
  Parsing infor from scraped files. Larz60+ 2 205 Apr-12-2019, 05:06 PM
Last Post: Larz60+
  Fetching and Parsing XML Data FalseFact 3 241 Apr-01-2019, 10:21 AM
Last Post: Larz60+
  HTML+CGI (if __name__ == '__main__') mahesh 6 280 Feb-19-2019, 04:21 PM
Last Post: snippsat
  Python + request from specific website - please help hoff1022 8 431 Feb-14-2019, 06:52 PM
Last Post: buran

Forum Jump:


Users browsing this thread: 1 Guest(s)