Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Webscraping with beautifulsoup
#4
(Aug-24-2023, 12:02 AM)cormanstan Wrote: May I ask why adding the user agent was so important?
User-Agent the website may block your requests because it knows you aren't a real user,like web-scraping or a bot.
Quote:Any suggestions how I can fix the error and call the make_graph function on gme_data and gme_revenue?
A test,so like this it should work see that most set regex=True if using a more that singel regex pattern in replace.
import pandas as pd

data = {
    'Date': ['2021/11/24', '2021/11/25', '2021/11/26', '2021/11/27'],
    'Price': ['$10,389', '$10,450', '$10,520', '$10,600'],
}

df = pd.DataFrame(data)
df['Date'] = pd.to_datetime(df['Date'])
df['Price'] = df['Price'].str.replace(r'\$|,', '', regex=True).astype(int)
>>> df
        Date  Price
0 2021-11-24  10389
1 2021-11-25  10450
2 2021-11-26  10520
3 2021-11-27  10600

# Works is now type int32
>>> df['Price'].max()
10600

>>> df.dtypes
Date     datetime64[ns]
Price             int32
dtype: object
 
Reply


Messages In This Thread
Webscraping with beautifulsoup - by cormanstan - Aug-23-2023, 12:04 AM
RE: Webscraping with beautifulsoup - by snippsat - Aug-23-2023, 07:09 AM
RE: Webscraping with beautifulsoup - by cormanstan - Aug-24-2023, 12:02 AM
RE: Webscraping with beautifulsoup - by snippsat - Aug-24-2023, 11:57 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Webscraping news articles by using selenium cate16 7 3,258 Aug-28-2023, 09:58 AM
Last Post: snippsat
  Webscraping returning empty table Buuuwq 0 1,438 Dec-09-2022, 10:41 AM
Last Post: Buuuwq
  WebScraping using Selenium library Korgik 0 1,071 Dec-09-2022, 09:51 AM
Last Post: Korgik
  How to get rid of numerical tokens in output (webscraping issue)? jps2020 0 1,977 Oct-26-2020, 05:37 PM
Last Post: jps2020
  Python Webscraping with a Login Website warriordazza 0 2,634 Jun-07-2020, 07:04 AM
Last Post: warriordazza
  Help with basic webscraping Captain_Snuggle 2 3,987 Nov-07-2019, 08:07 PM
Last Post: kozaizsvemira
  Can't Resolve Webscraping AttributeError Hass 1 2,341 Jan-15-2019, 09:36 PM
Last Post: nilamo
  How to exclude certain links while webscraping basis on keywords Prince_Bhatia 0 3,269 Oct-31-2018, 07:00 AM
Last Post: Prince_Bhatia
  Webscraping homework Ghigo1995 1 2,681 Sep-23-2018, 07:36 PM
Last Post: nilamo
  Intro to WebScraping d1rjr03 2 3,488 Aug-15-2018, 12:05 AM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020