Python Forum
Thread Rating:
  • 1 Vote(s) - 3 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web Crawler Not Working
#12
(Feb-03-2017, 08:43 AM)wavic Wrote: I run a copy of the code and got this:
The error is on you side,
to make sure i tested in Python 3.6 with virtual environment(new install of BeautifulSoup and Requests).
It do work,make sure your libraries are updated or run virtual environment(with new install).

Break it down,to see where the problem is.
First as eg,do you get source code.
import requests

url = 'http://theiconic.com.au/mens-clothing-tshirts-singlets/?page=1'
source_code = requests.get(url)
plain_text = source_code.text
print(plain_text[:90])
Output:
<!DOCTYPE html> <!--[if IE 7]>  <html xmlns:ng="http://angularjs.org" class="ie7" lang="en
Reply


Messages In This Thread
Web Crawler Not Working - by chrisdas - Jan-24-2017, 12:54 PM
RE: Web Crawler Not Working - by scriptso - Feb-02-2017, 11:06 PM
RE: Web Crawler Not Working - by wavic - Feb-02-2017, 11:23 PM
RE: Web Crawler Not Working - by scriptso - Feb-02-2017, 11:25 PM
RE: Web Crawler Not Working - by wavic - Feb-03-2017, 12:39 AM
RE: Web Crawler Not Working - by scriptso - Feb-03-2017, 01:18 AM
RE: Web Crawler Not Working - by wavic - Feb-03-2017, 01:34 AM
RE: Web Crawler Not Working - by scriptso - Feb-03-2017, 01:47 AM
RE: Web Crawler Not Working - by wavic - Feb-03-2017, 03:33 AM
RE: Web Crawler Not Working - by snippsat - Feb-03-2017, 05:59 AM
RE: Web Crawler Not Working - by wavic - Feb-03-2017, 08:43 AM
RE: Web Crawler Not Working - by snippsat - Feb-03-2017, 09:21 AM
RE: Web Crawler Not Working - by wavic - Feb-03-2017, 09:48 AM
RE: Web Crawler Not Working - by scriptso - Feb-06-2017, 10:45 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  stucked with my crawler jviure 0 1,812 Apr-15-2020, 10:47 AM
Last Post: jviure
  Web Crawler help Mr_Mafia 2 2,729 Apr-04-2020, 07:20 PM
Last Post: Mr_Mafia
  Crawler Telegram sysx90 0 3,009 Nov-30-2019, 04:32 PM
Last Post: sysx90
  web crawler problems kid_with_polio 3 3,058 Jun-15-2019, 12:33 AM
Last Post: metulburr
  Web Crawler help takaa 39 34,069 Apr-26-2019, 12:14 PM
Last Post: stateitreal
  Python - Why multi threads are not working in this web crawler? ratanbhushan 1 3,479 Nov-17-2017, 05:21 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020