Python Forum
Scrape Multiple items from a webpage
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scrape Multiple items from a webpage
#1
i am trying to scrape image and table from a wikipedia page and write it into csv but i am confused that how to club them together and write this data into csv.

below are my codes
from urllib.request import urlopen
from bs4 import BeautifulSoup

url = "https://en.wikipedia.org/wiki/Kevin_Bacon"
html = urlopen(url)
soup = BeautifulSoup(html, "html.parser")

newfile = "Newlyout.csv"
f = open(newfile, "w")
Headers = "Year, Association, Category, Nominated, Results, Imagelink\n"
f.write(Headers)

soup1 = soup.find_all("img")
for i in soup1:
    Image = i['src']
    
    #ddprint(Image['src'])
    soup3 = soup.find("table", {"class":"wikitable sortable"})
    for tag in soup3.find_all("tr"):
        cell = tag.find_all("td")
        
        if len(cell) == 5:
            Year = cell[0].find(text=True)
            Association = cell[2].find(text=True)
            Category = cell[3].find(text=True)
            Nominated = cell[4].find(text=True)
            Results = cell[4].find(text=True)
            f.write("{}".format(Year)+ ",{}".format(Association)+ ",{}".format(Category) + ",{}".format(Nominated) + ",{}".format(Results)+ ",{}".format(Image)+"\n")
            
f.close()
i got it solved till here but it is repeating the data..and in images there are multiple images in one single cell....all i need table and against it all images in that page..
Reply
#2
Quote:
soup1 = soup.find_all("img")
for i in soup1:
    Image = i['src']
     
    #ddprint(Image['src'])
    soup3 = soup.find("table", {"class":"wikitable sortable"})

So for every image on the page... find all the tables with a certain class, and then do more stuff.
Those sound like two different things.
Reply
#3
Problem is that i am not able to club them them together into csv file...but it is repeating the data..and in images there are multiple images in one single cell....all i need table and against it all images in that page..
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Scrape table from multiple pages Nhattanktnn 1 823 Jun-07-2023, 09:35 AM
Last Post: Larz60+
  scrape data 1 go to next page scrape data 2 and so on alkaline3 6 5,087 Mar-13-2020, 07:59 PM
Last Post: alkaline3
  Pass multiple items from one parse to another using Scrapy nazmulfinance 2 4,556 Jan-23-2020, 06:44 PM
Last Post: nazmulfinance
  Need advice how to scrape a Chinese webpage omar 2 1,998 Nov-21-2019, 12:30 PM
Last Post: snippsat
  Scrape multiple urls LXML santdoyle 1 3,514 Oct-26-2019, 09:53 PM
Last Post: snippsat
  How to create Flask-Stripe Checkout and Charge for the multiple items Antares 4 5,207 Jul-05-2019, 10:20 AM
Last Post: Antares
  display multiple sensors on webpage python flask jinja pascale 6 5,191 Jan-29-2019, 10:10 AM
Last Post: pascale
  Multiple input box in a webpage have same XPATH sumandas89 3 5,089 Jul-19-2018, 08:41 AM
Last Post: buran
  Scrape multiple lines with regex greetings 2 3,021 Jul-04-2018, 09:09 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020