Python Forum
urlib - to use or not to use ( for web scraping )?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urlib - to use or not to use ( for web scraping )?
#38
Thank you.

Now I'm trying to make a code that downloads images from a page and as you can imagine...it doesn't go that well
import requests
from bs4 import BeautifulSoup
import shutil

html = requests.get("http://www.pythonscraping.com", stream=True)
bsObj = BeautifulSoup(html.content, 'html.parser')
imageLocation = bsObj.find("a", {"id":"logo"}).find("img")["src"]
with open('img.jpg', 'wb') as out_file:
	shutil.copyfileobj(imageLocation, out_file)
Error:
Traceback (most recent call last): File "C:\Python36\kodovi\crawler3.py", line 9, in <module> shutil.copyfileobj(imageLocation, out_file) File "C:\Python36\lib\shutil.py", line 79, in copyfileobj buf = fsrc.read(length) AttributeError: 'str' object has no attribute 'read'
this is the urlib code from the book that I'm trying to transform:
from urllib.request import urlretrieve
from urllib.request import urlopen
from bs4 import BeautifulSoup
html = urlopen("http://www.pythonscraping.com")
bsObj = BeautifulSoup(html)
imageLocation = bsObj.find("a", {"id": "logo"}).find("img")["src"]
urlretrieve (imageLocation, "logo.jpg")
Reply


Messages In This Thread
RE: urlib - to use or not to use ( for web scraping )? - by Truman - Dec-11-2018, 11:49 PM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020