Feb-12-2018, 07:33 AM
(This post was last modified: Feb-12-2018, 08:02 AM by basheerkharoti.)
The Scrapy application just works fine when run from the command line and within the same directory. e.x
When we navigate to the folder and run a spider then that just works fine. While the same command is being used from another directory/folder then it just seems to throw error.
Following is the command that we use from a PHP application
However I've confirmed the path
Thank you
Ah just solved it by
When we navigate to the folder and run a spider then that just works fine. While the same command is being used from another directory/folder then it just seems to throw error.
Following is the command that we use from a PHP application
exec('scrapy runspider G:/thevillages/thevillages/spiders/village_spider.py 2>&1', $output);The errors are as follows
Quote:
0 => "2018-02-12 12:29:08 [scrapy.utils.log] INFO: Scrapy 1.4.0 started (bot: scrapybot)"
1 => "2018-02-12 12:29:08 [scrapy.utils.log] INFO: Overridden settings: {'SPIDER_LOADER_WARN_ONLY': True}"
2 => "Usage"
3 => "====="
4 => " scrapy runspider [options] <spider_file>"
5 => ""
6 => "runspider: error: Unable to load 'G:/thevillages/thevillages/spiders/village_spider.py': No module named thevillages.items"
However I've confirmed the path
>>> os.path.exists('G:/thevillages/thevillages/spiders') True >>> os.path.exists('G:/thevillages/thevillages') True >>> os.path.exists('G:/thevillages') True >>> os.path.exists('G:/thevillages/thevillages/spiders/village_spider.py') TrueI don't really have much experince with python but I do asked on most of the forums and they did advice to ask question on the python-fourm.
Thank you
Ah just solved it by
exec('cd G:/thevillages/thevillages/spiders/ && scrapy runspider village_spider.py &', $output);