Python Forum
Webscrape using RPi and SQlite database, always write the last value in database
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Webscrape using RPi and SQlite database, always write the last value in database
#1
Hi!
I have some code that will webscrape a value and store it in a SQlite database.
My problem is that only the last value is stored in the database.
When I do the same with windows, it stores all values in the database when webscraping.
I dont know why it only stores the last value.


webscrapeall.py
import requests
import sqlite3
import datetime 

from bs4 import BeautifulSoup
URL = "http://ludvikasegel.com/wx/cloudbase.asp"
page = requests.get(URL)
soup = BeautifulSoup(page.content, "html.parser")
molnbas = soup.find_all("div", class_="cloudbase")
print (molnbas[0].text.strip())


with sqlite3.connect('database.db') as con, open("/home/admin/schema.sql") as f:
    con.executescript(f.read())


    timestamp = datetime.datetime.now().isoformat()  # timestamp, change to whatever you want
    desired_value = molnbas[0].text.strip()
    

with sqlite3.connect("/home/admin/database.db") as sqlite_connection:
    sqlite_connection.execute(
        "INSERT INTO table_name VALUES (?, ?)",
        (timestamp, desired_value)
    )
schema.sql
DROP TABLE IF EXISTS table_name;

CREATE TABLE table_name (
    timestamp TEXT,
    value INTEGER
);
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Webscrape script, add to run every day at a specific time jjcooper 2 292 Jun-07-2024, 03:46 AM
Last Post: sawtooth500
  Django: View is unable to find attributes of database model pythonpaul32 1 793 May-20-2024, 05:38 AM
Last Post: tahirahmedd
  Flask: sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) database is locked pythonpaul32 1 2,383 Apr-04-2023, 07:44 AM
Last Post: Larz60+
  Flask and SQLAlchemy question: Database is being created but tables aren't adding pythonpaul32 3 5,373 Feb-07-2023, 10:48 AM
Last Post: pythonpaul32
  Flask/non-flask database sharing MorganSamage 2 1,326 Feb-03-2023, 12:05 PM
Last Post: MorganSamage
  Save JSON data to sqlite database on Django Quin 0 3,014 Mar-26-2022, 06:22 PM
Last Post: Quin
  Python Desktop Application that will connect o Azure SQL Database thewolf 7 4,812 Feb-26-2021, 01:33 AM
Last Post: thewolf
Question Selenium Screenshots store in Database Nuwan16 3 2,130 Oct-05-2020, 02:55 PM
Last Post: Nuwan16
  User Input to mySQL database newbie1 3 4,360 Aug-26-2020, 10:42 AM
Last Post: newbie1
Question wkhtmltoimage generated jpeg save to Database using mongoengine Madoo 2 3,203 Aug-18-2020, 03:42 PM
Last Post: Madoo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020