Python Forum
fetching, parsing data from Wikipedia
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
fetching, parsing data from Wikipedia
#1
hello dear python-experts, good day. Smile



this scraper fetches wikipedia pages



it is a nice little scraper - it ...:

import requests
import urllib.request
import time
from bs4 import BeautifulSoup
import numpy as np
import pandas as pd
from urllib.request import urlopen
url = 'https://en.wikipedia.org/wiki/List_of_cities_by_sunshine_duration'
html = urlopen(url) 
soup = BeautifulSoup(html, 'html.parser')
these few lines fetch data . but i guess that i need more.

i am going to add some

find_all('table') ## that helps me to scan the entire document to look for the following tag <table>
and the following

tables = soup.find_all('table')
can i do this like so?
Reply


Messages In This Thread
fetching, parsing data from Wikipedia - by apollo - May-05-2021, 06:12 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Fetching Images from DB in Django Dexty 2 1,733 Mar-15-2024, 08:43 AM
Last Post: firn100
Question Scraping Wikipedia Article (Name in 1 column & URL in 2nd column) ->CSV! Anyone? BrandonKastning 4 2,046 Jan-27-2022, 04:36 AM
Last Post: Larz60+
  Logic behind BeautifulSoup data-parsing jimsxxl 7 4,327 Apr-13-2021, 09:06 AM
Last Post: jimsxxl
  Need help scraping wikipedia table bborusz2 6 3,268 Dec-01-2020, 11:31 PM
Last Post: snippsat
  table from wikipedia flow50 5 5,462 Jul-01-2019, 07:12 PM
Last Post: snippsat
  Fetching and Parsing XML Data FalseFact 3 3,281 Apr-01-2019, 10:21 AM
Last Post: Larz60+
  how to make my product description fetching function generic? PrateekG 10 6,097 Jun-29-2018, 01:03 PM
Last Post: PrateekG
  Getting 'list index out of range' while fetching product details using BeautifulSoup? PrateekG 8 8,189 Jun-06-2018, 12:15 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020