Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urllib request urlopen?
#4
(Mar-26-2018, 11:39 AM)nutgut Wrote: Would be fun to hear what others think about the solution.
The solution is okay,but he dos a lot error checking that can be confusing.
It it's simplest form,here download a CSV from web.
Always use Requests and not urllib.
import requests

url = 'http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv'
url_get = requests.get(url)
# Download csv
with open('sample.csv', 'wb') as f_out:
    f_out.write(url_get.content)
Example parse out that link from the website,and the use it.
import requests
from bs4 import BeautifulSoup

url_csv = 'http://www.sample-videos.com/download-sample-csv.php'
url = requests.get(url_csv)
soup = BeautifulSoup(url.content, 'lxml')
h1 = soup.find('h1')
print(h1.text)
print('------------')
site = soup.find('a', class_="navbar-brand")
link = soup.find('a', class_="download_csv")
adress_csv = f"{site.get('href')}/{link.get('href')}"
print(adress_csv)

# Download csv
download_link = requests.get(adress_csv)
csv_url_name = adress_csv.split('/')[-1]
print(csv_url_name)
with open(csv_url_name, 'wb') as f_out:
    f_out.write(download_link.content)
Output:
Download Sample CSV ------------ http://www.sample-videos.com/csv/Sample-Spreadsheet-10-rows.csv Sample-Spreadsheet-10-rows.csv
Reply


Messages In This Thread
urllib request urlopen? - by nutgut - Mar-24-2018, 07:30 PM
RE: urllib request urlopen? - by Larz60+ - Mar-24-2018, 08:53 PM
RE: urllib request urlopen? - by nutgut - Mar-26-2018, 11:39 AM
RE: urllib request urlopen? - by snippsat - Mar-26-2018, 05:15 PM
RE: urllib request urlopen? - by nutgut - Apr-14-2018, 01:12 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Getting from <td> tag by using urllib,Beautifulsoup KuroBuster 2 2,088 Aug-20-2021, 07:53 AM
Last Post: KuroBuster
  Can urlopen be blocked by websites? peterjv26 2 3,440 Jul-26-2020, 06:45 PM
Last Post: peterjv26
  Beginner: urllib error tomfry 7 6,625 May-03-2020, 04:35 AM
Last Post: Larz60+
  SSLCertVerificationError using urllib (urlopen) FalseFact 1 5,936 Mar-31-2019, 08:34 AM
Last Post: snippsat
  Error: module 'urllib' has no attribute 'urlopen' mitmit293 2 15,128 Jan-29-2019, 02:32 PM
Last Post: snippsat
  [Errno11004] Get addrinfo failed with urlopen prashanth0988 2 13,878 Aug-02-2018, 01:41 PM
Last Post: iiooii
  urllib urlopen getting error 400 on 1 specific page glidecode 4 4,179 Mar-01-2018, 11:01 PM
Last Post: glidecode
  urllib2.urlopen() user agent header Skaperen 8 12,733 Jul-14-2017, 05:36 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020