Hello forum!
First post and starting it of by making yself look extra nubZ! lol So I just reached (...yesterday infact ) first year of complte diving into everything python... If you check my I have a repo of 99.98% python forks and personal project and its about 150ish deep? If embarked on this journey to meet and surpass the reuirements for various certifications In a month ill be completibng/enrolling (WebSec heavy)... past 4 months its been nothing but scrapping and data analytics...
#Just made a quick dirty scrapy script Ill use for what Im trying to get across
Self taught, theres no stock module I have issues with but the CSV module... I have a really hard time understanding the seemingly basic logic behind using.. Here the flow of what Im trying to do in an example scenario..
#Which I know probly using OS I can get it to just run in current direct or w.e
... but also doing extra editing to the output has become a 2 week wtf issue... Extra edit fields?
1. To add "#EXTM3U" as the new hear #all attempts have either looped "#EXTM3U" along or error out
2 Add "#EXTINF:0," before every iteration in "vidName" #Issue being that coma after the 0 though not the important part cus it really doesnt need it to function as a M3U playlist
DESIRED OUTPUT!
ALSO, There has to be a way to include all this into on script right?? Help a noob out?! Point me to the right direction?! lol I am googling and doinf csv module tuts but could use a hand
First post and starting it of by making yself look extra nubZ! lol So I just reached (...yesterday infact ) first year of complte diving into everything python... If you check my I have a repo of 99.98% python forks and personal project and its about 150ish deep? If embarked on this journey to meet and surpass the reuirements for various certifications In a month ill be completibng/enrolling (WebSec heavy)... past 4 months its been nothing but scrapping and data analytics...
#Just made a quick dirty scrapy script Ill use for what Im trying to get across
Self taught, theres no stock module I have issues with but the CSV module... I have a really hard time understanding the seemingly basic logic behind using.. Here the flow of what Im trying to do in an example scenario..
#scrapy script
# -*- coding: utf-8 -*-
import scrapy
class PyvsrcSpider(scrapy.Spider):
name = "pyVSrc"
allowed_domains = ["pyvideo.org"]
start_urls = ['ht tp: //pyv ideo. org /t ag/ tuto ria l/'] #not been proved to no be spammer on forum
def parse(self, response):
# follow links to author pages
# item = tagitems()
# item['tag2'] = response.css('div.headline h1::text').re_first('\w+$');
for href in response.css('div.thumb a::attr(href)').extract():
yield scrapy.Request(response.urljoin(href),
callback=self.parse_mainsrc)
# Reason for the second def is because I use this as a template for qyuickjk and dirty
# scraping and where this comen is would lie a pagination script... #muchmess
def parse_mainsrc(self, response):
yield {
'vidName': response.css('h2.entry-title a::text').extract(),
'link2vid': response.css('iframe').xpath('@src').extract(),
# 'tags': response.css('div.video-tags a::text').extract()
}
##### csvrewrite.py
_author_ = '...not me
import csv
with open('pyvids_tuts.csv') as f:
reader = csv.reader(f)
new_data = []
for row in reader:
if reader.line_num == 1:
continue
print(row[1] + "\n" + row[0])Outputing results into csv, as you can see, output two tables.. of which the I run a second script in order to parse all items.. The idea is to have iterated 'NAME,TITLE' so it out puts as... #and exclude the headersOutput:vidName1
link2vid1
vidName2
link2vid2
vidName2
link2vid2#Which I know probly using OS I can get it to just run in current direct or w.e
... but also doing extra editing to the output has become a 2 week wtf issue... Extra edit fields?
1. To add "#EXTM3U" as the new hear #all attempts have either looped "#EXTM3U" along or error out
2 Add "#EXTINF:0," before every iteration in "vidName" #Issue being that coma after the 0 though not the important part cus it really doesnt need it to function as a M3U playlist
DESIRED OUTPUT!
Output:#EXTM3U
#EXTINF:0,vidName
link2vid1
#EXTINF:0,vidName2
link2vid2
#EXTINF:0,vidName3
link2vid3ALSO, There has to be a way to include all this into on script right?? Help a noob out?! Point me to the right direction?! lol I am googling and doinf csv module tuts but could use a hand
