Python Forum
Scrapy-cut: Advanced Cookiecutter Scrapy Templating
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrapy-cut: Advanced Cookiecutter Scrapy Templating
#1
This post is to inquire any references or resources is... any pointers  to the following project scope whilst using cookie cutter framework.

can check it out at https://github.com/3rawkz/scrapy-cut


Please review the git repository. You might get a sense of what I'm trying to accomplish, I literally just threw up that Reba the past half-hour and no I don't feel it's up to a draft completion my question is the following.

 Using Jinja,is it possible have multiple templates but in some sort of dictionary be able to call just one? For example I know the end official documents for cookiecutter I first learned that while creating the cookiecutter-pypackage, the license content is located in one text file, of which using else if statement creates a dictionary which is defined  in the json file... Is similar process for python template? Or is it creating a text file with the python code and then figuring out how to do code to create the spider?

Secondly, if I refer you to the "demo" spider code snippet...

yield {

           'name': "{{cookiecutter.item_name_selector}}",

           'event': "{{cookiecutter.item_event_selector}}",

           'vidurl': "{{cookiecutter.item_vidurl_selector}}",

           'tags': "{{cookiecutter.item_tags_selector}}",


           }
 

As you can Imagine... (or if you are slightly familiar with cookiecutter would have check out the json file), the variables are defined and in this scrapy project not using items.py for the sake of starting easy... In the given code snip while using cookicutter... Is there some way I can loop to input x desired amount of items and the "Item selector" to add as many as I need?
Reply
#2
Bad news to name a dictionary yield, that is reserved for generators
Reply
#3
(Feb-02-2017, 07:51 PM)Larz60+ Wrote: Bad news to name a dictionary yield, that is reserved for generators

Larz, youre always the first to ever reply... your dedication on here is admirable! MORE BADGES 4 U! lol.. And yes! that correct... The block of code to im referring to in my Second question, as to the how-to have a loop to create the the yield is just the item names an the responsee selector, inside the actual spider the yield {....} stays... like had-coded
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Scrapy does not show all data in iteration loop georgekasa 0 1,948 Jul-31-2021, 09:10 AM
Last Post: georgekasa
Lightbulb Hypertag. New language for HTML templating w/ Django support mwojnars 0 1,722 Apr-06-2021, 12:53 PM
Last Post: mwojnars
  pip install scrapy fail Barak 10 5,404 Aug-31-2020, 07:45 AM
Last Post: Barak
  Scrapy-Splash installation on windows 10 dsuk 0 2,201 Aug-27-2020, 05:45 AM
Last Post: dsuk
  Python Scrapy Date Extraction Issue tr8585 1 3,191 Aug-05-2020, 04:32 AM
Last Post: tr8585
  Python Scrapy tr8585 2 2,286 Aug-04-2020, 04:11 AM
Last Post: tr8585
  Scraping using Scrapy userskipper 0 2,026 Jul-23-2020, 02:20 PM
Last Post: userskipper
  A problem with installing scrapy Truman 17 9,328 May-24-2020, 07:55 PM
Last Post: Truman
  Merging selenium and scrapy Veakyp 0 1,469 Apr-21-2020, 09:58 AM
Last Post: Veakyp
  How to crawl schema markup data type using scrapy? Nuwan16 1 3,017 Mar-31-2020, 03:42 PM
Last Post: stranac

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020