Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Problem parsing website html file
#1
Hi Guys,

I am trying to parse this web link (http://insight.toba.fun/address/TVAWF6eD...sigfWK2oRB) using Python3 and requests. When we click the link it loads the details of the account. However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link. I tried putting a delay but didn't work. I'm unable to get the account the info using python. Can anyone point out a solution for this?
I tried using selenium webdriver, but I'm unable to find the search box in the html.

import requests
url='http://insight.toba.fun/address/TVAWF6eDfW9MRdLitCb7ag41sigfWK2oRB'
r = requests.get(url,verify=False)
print(r.text)
Quote
#2
Quote:However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link.
You are not saving the file.
add after line 3:
with open('yourfilename', 'w') as f:
    f.write(r.text)
(of course, replace 'yourfilername' with a valid .html file name)
Quote
#3
(May-01-2018, 11:04 AM)Larz60+ Wrote:
Quote:However, when calling from requests, it does not generate the HTML page that can be obtained by clicking the link.
You are not saving the file.
add after line 3:
with open('yourfilename', 'w') as f:
    f.write(r.text)
(of course, replace 'yourfilername' with a valid .html file name)

The OP probably meant - page is not rendered properly because of Javascript. Try https://html.python-requests.org/ which can render the page with Chromium.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Python3 + BeautifulSoup4 + lxml (HTML -> CSV) - How to loop to next HTML/new CSV Row BrandonKastning 0 64 Mar-22-2020, 06:10 AM
Last Post: BrandonKastning
  Parsing based on variables in the website nikos48 3 211 Jan-19-2020, 03:41 PM
Last Post: nikos48
  Help on parsing simple text on HTML amaumox 5 288 Jan-03-2020, 05:50 PM
Last Post: amaumox
  Create .exe file for Python flask website. vintysaw 4 1,061 Nov-18-2019, 07:56 AM
Last Post: tonycstech
  Sending file html ? JohnnyCoffee 3 338 Sep-06-2019, 04:32 PM
Last Post: snippsat
  download pdf file from website m_annur2001 1 536 Jun-21-2019, 05:03 AM
Last Post: j.crater
  XML parsing and generating HTML page Python 3.6 Madhuri 2 1,212 Aug-24-2018, 02:48 PM
Last Post: snippsat
  Reading a html file peterl 4 1,053 Aug-20-2018, 03:16 PM
Last Post: peterl
  html to text problem Kyle 4 1,928 Apr-27-2018, 09:02 PM
Last Post: snippsat
  Multi-Threaded Alexa Website Ranker Problem - All Threads Doing Same Task digitalmatic7 0 813 Feb-28-2018, 09:21 AM
Last Post: digitalmatic7

Forum Jump:


Users browsing this thread: 1 Guest(s)