Python Forum
Web App That Request Data from Another Web Site every 12-hours
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web App That Request Data from Another Web Site every 12-hours
#11
You need to save it somewhere first. If it isn't that important to you, you can store it in memory and make it a global variable, and then pass it to the template like how was shown earlier with the location parameter.

Something like this:
from flask import Flask, render_template, jsonify
from apscheduler.schedulers.background import BackgroundScheduler 
import random
import requests
 
app = Flask(__name__)

content = None

def parse_func():
    global content

    response = requests.get('https://nghttp2.org/httpbin/get')
    r = response.json()
    lst = [r['url'], r['origin']]
    rand_value = random.choice(lst) 
    content = rand_value
 
@app.route("/")
def template():
    global content
    return render_template('sh2.html', location=content)
 
if __name__ == '__main__':
    scheduler = BackgroundScheduler()
    scheduler.add_job(parse_func, 'interval', seconds=15)
    scheduler.start()
    app.run(debug=True)
Reply
#12
(Sep-24-2018, 04:23 PM)nilamo Wrote: You need to save it somewhere first. If it isn't that important to you, you can store it in memory and make it a global variable, and then pass it to the template like how was shown earlier with the location parameter. Something like this:
from flask import Flask, render_template, jsonify from apscheduler.schedulers.background import BackgroundScheduler import random import requests app = Flask(__name__) content = None def parse_func(): global content response = requests.get('https://nghttp2.org/httpbin/get') r = response.json() lst = [r['url'], r['origin']] rand_value = random.choice(lst) content = rand_value @app.route("/") def template(): global content return render_template('sh2.html', location=content) if __name__ == '__main__': scheduler = BackgroundScheduler() scheduler.add_job(parse_func, 'interval', seconds=15) scheduler.start() app.run(debug=True)
Thank you very much! Is it ok to use global variables for such applications or there are any other approaches?
Reply
#13
If the server restarts, do you still need access to the data? You'll lose it if it's a global, until the scheduled part runs again.

I don't know what kind of data it is, but databases are built to store things.
Reply
#14
Yes I need to access it. So, I have to build a query for database with SQLAlchemy or something ?
Reply
#15
That depends on what the data is. It might make more sense to just store it in a plain text file, and read it once when the server starts up. It's really hard to say what you "should" do, because we really don't have any idea what you're doing, what sort of traffic you're expecting, what the data is, etc.

I mean, you could just do it, and see if you're happy with the performance you're getting.
Reply
#16
(Sep-24-2018, 07:56 PM)jomonetta Wrote: Yes I need to access it. So, I have to build a query for database with SQLAlchemy or something ?
You should use Flask-SQLAlchemy it's the common way for database connection in Flask.
It's elegant made and can plug into database of choice.
SQLite is a good choice for most small to middel large task.

To give a demo that build on what we have here,
so now values is send to client side,then take values back to server side.
Where connect to SQLite and place values in database location.db(one file and portable as it's SQLite).
from flask import Flask, render_template, jsonify, url_for, request, redirect
from flask_sqlalchemy import SQLAlchemy
from apscheduler.schedulers.background import BackgroundScheduler 
import random
import requests

app = Flask(__name__)
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:///location.db'
db = SQLAlchemy(app)

class Loc(db.Model): 
    id = db.Column(db.Integer, primary_key=True)   
    content = db.Column(db.String(4096), nullable=False)

def parse_func():
    response = requests.get('https://nghttp2.org/httpbin/get')
    r = response.json()
    lst = [r['url'], r['origin']]
    rand_value = random.choice(lst)
    #print(rand_value) 
    return(rand_value)    

@app.route("/foo")
def foo():    
    parse = parse_func()    
    loc = Loc(content=parse)                   
    db.session.add(loc)
    db.session.commit()
    return render_template('sh2_base.html', location=parse)  

@app.route('/db_save')
def db_save():
    my_var = request.args.get('my_var', None)
    loc = Loc(content=my_var)                   
    db.session.add(loc)
    db.session.commit()  
    #print(my_var)
    return redirect(url_for('foo'))

if __name__ == '__main__':
    scheduler = BackgroundScheduler()
    scheduler.add_job(parse_func, 'interval', seconds=15)
    scheduler.start()
    app.run(debug=True)
sh2_base.html:
<!doctype html>
<html>
<head>
  <title>Some title</title>
  <link rel=stylesheet type=text/css href="{{ url_for('static', filename='css/style.css') }}" />
</head>
<body>
  <a href="{{ url_for('db_save', my_var=location) }}">Send my_value</a>
  <p>{{ location }}</p>
</body>
<script> 
    function timedRefresh(timeoutPeriod) {
      setTimeout("location.reload(true);",timeoutPeriod);
    }    
    window.onload = timedRefresh(15000);   
</script>
</html>
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How to scraping data from dinamic site sergio21124444 2 646 Nov-08-2023, 12:43 PM
Last Post: sergio21124444
  POST request with form data issue web scraping hoff1022 1 2,649 Aug-14-2020, 10:25 AM
Last Post: kashcode
  Scraping a dynamic data-table in python through AJAX request filozofo 1 3,823 Aug-14-2020, 10:13 AM
Last Post: kashcode
  How to retrieve data from site ROHK 2 2,402 Mar-01-2019, 12:26 PM
Last Post: ROHK
  Mechanize and BeautifulSoup read not correct hours vaeVictis 5 4,350 Jan-15-2019, 01:27 PM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020