Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
A problem with installing scrapy
#8
Thank you. I have windows on 64 bits which confused me.

I installed scrapy too but get this error
Error:
Traceback (most recent call last): File "C:\Python36\kodovi\scrap.py", line 1, in <module> import scrapy File "C:\Python36\lib\site-packages\scrapy\__init__.py", line 34, in <module> from scrapy.spiders import Spider File "C:\Python36\lib\site-packages\scrapy\spiders\__init__.py", line 10, in < module> from scrapy.http import Request File "C:\Python36\lib\site-packages\scrapy\http\__init__.py", line 8, in <modu le> from scrapy.http.headers import Headers File "C:\Python36\lib\site-packages\scrapy\http\headers.py", line 4, in <modul e> from scrapy.utils.python import to_unicode File "C:\Python36\lib\site-packages\scrapy\utils\python.py", line 14, in <modu le> from scrapy.utils.decorators import deprecated File "C:\Python36\lib\site-packages\scrapy\utils\decorators.py", line 4, in <m odule> from twisted.internet import defer, threads File "C:\Python36\lib\site-packages\twisted\internet\defer.py", line 31, in <m odule> from twisted.python import lockfile, failure File "C:\Python36\lib\site-packages\twisted\python\lockfile.py", line 46, in < module> import pywintypes File "C:\Python36\lib\site-packages\win32\lib\pywintypes.py", line 136, in <mo dule> __import_pywin32_system_module__("pywintypes", globals()) File "C:\Python36\lib\site-packages\win32\lib\pywintypes.py", line 39, in __im port_pywin32_system_module__ for suffix_item in imp.get_suffixes(): AttributeError: module 'imp' has no attribute 'get_suffixes'
when trying to execute code:
import scrapy
from scrapy.crawler import CrawlerProcess

class PythonEventsSpider(scrapy.Spider):
	name = 'pythoneventsspider'
	
	start_urls = ['https://www.python.org/events/python-events/',]
	found_events = []
	
	def parse(self, response):
		for event in response.xpath('//ul[contains(@class, "list-recentevents")]/li'):
			event_details = dict()
			event_details['name'] = event.xpath('h3[@class="event-title"]/a/text()').extract_first()
			event_details['location'] = event.xpath('p/span[@class="event-location"]/text()').extract_first()
			event_details['time'] = event.xpath('p/time/text()').extract_first()
			self.found_events.append(event_details)
			
if __name__ == "__main__":
	process = CrawlerProcess({ 'LOG_LEVEL': 'ERROR'})
	process.crawl(PythonEventsSpider)
	spider = next(iter(process.crawlers)).spider
	process.start()
	
	for event in spider.found_events:
		print(event)
As per error message it looks that there are some problems with packages.

Also wondering should I use scrapy at all. This stuff can be done with requests and bs4 in less lines.
Reply


Messages In This Thread
A problem with installing scrapy - by Truman - May-21-2019, 12:57 AM
RE: A problem with installing scrapy - by snippsat - May-21-2019, 05:10 AM
RE: A problem with installing scrapy - by heiner55 - May-21-2019, 05:11 AM
RE: A problem with installing scrapy - by Truman - May-21-2019, 01:42 PM
RE: A problem with installing scrapy - by snippsat - May-21-2019, 02:09 PM
RE: A problem with installing scrapy - by Truman - May-21-2019, 10:08 PM
RE: A problem with installing scrapy - by snippsat - May-21-2019, 10:43 PM
RE: A problem with installing scrapy - by Truman - May-21-2019, 11:09 PM
RE: A problem with installing scrapy - by snippsat - May-22-2019, 10:16 PM
RE: A problem with installing scrapy - by Truman - May-23-2019, 11:46 PM
RE: A problem with installing scrapy - by Truman - May-24-2019, 11:41 PM
RE: A problem with installing scrapy - by snippsat - May-25-2019, 07:28 AM
RE: A problem with installing scrapy - by Truman - May-25-2019, 11:25 PM
RE: A problem with installing scrapy - by heiner55 - May-29-2019, 02:49 AM
RE: A problem with installing scrapy - by Truman - May-29-2019, 10:34 PM
RE: A problem with installing scrapy - by heiner55 - May-30-2019, 03:52 AM
RE: A problem with installing scrapy - by bishwasbh - May-16-2020, 05:21 AM
RE: A problem with installing scrapy - by Truman - May-24-2020, 07:55 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Old problem back hunting me installing Blue Dog 4 3,502 Jun-02-2018, 09:49 AM
Last Post: Blue Dog
  Scrapy-cut: Advanced Cookiecutter Scrapy Templating scriptso 2 4,806 Feb-02-2017, 07:57 PM
Last Post: scriptso

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020