Python Forum
Take data from web page problem
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Take data from web page problem
#1
Best regards, I kindly ask you, I have a problem and I have just started using Python, I have spent hours trying to solve this little question but I can't get it unstuck
import requests
from bs4 import BeautifulSoup

url = "https://www.sito web"
r = requests.get(url)
soup=BeautifulSoup(r.text, 'html.parser')
print(soup.title.text)
in this way, in a very curious way, he takes exactly the data I need from a web page even though I haven't explained to him in detail where it is, but I would like that once the data has been retrieved, which is something like 0.00000120, he would make a comparison with a value fixed set by me for example 0.00000110 and if the value retrieved from the web page is greater than 0.00000110 then write to me the value otherwise write to me "the value is lower!
I have tried in many ways but I get errors regarding float and string and unicode which does not can be put together.
thank you
Reply
#2
(Oct-26-2023, 01:57 PM)codeweak Wrote: in a very curious way, he takes exactly the data I need from a web page even though I haven't explained to him in detail where it is
You did - soup.title.text - you take the page title (i.e. the title that you see in browser tab). You don't scrape content from the page body (if that is what you think)

page_data = float(soup.title.text)
threshold = 0.00000110

if page_data > threshold:
    print(f"It's greater than {threshold}")
else:
    print(f"{it's less than or equal to {threshold}")
Now, because these are float numbers, you should be aware for some Floating Point Arithmetic: Issues and Limitations and Is floating point math broken?
codeweak likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thumbs Up 
[Image: 1-Copia.png]

Thank You so much I think that the problem now is the formatting of the data it recovers, in fact you can see in the image the type of data it perhaps also has grids.
I think that to get help in the right way I have to tell you exactly what I want to do. I have to take at this <div class="subPrice"">€,00000112</div> in this page
https://www.binance.com/it/trade/PEPE_US...&type=spot

Next I tell the program if this data is greater of (for example 0.00000100) then write it to me and tell me how much it is if instead the data you extract is less than 0.00000100 then write to me that the price has not risen. this must happen in a recursive manner until I decide to close the program, but the program must remain active in the background and warn me if the price rises, at this moment I want to add an alarm system sound if the price has risen beyond a certain value (e.g. 0.00000180)

Thank You
Reply
#4
Is convenient and strange that price is in title of page.
This is a site page that typically would need Selenium to parse any of the content.
So then can do it like this.
import requests
from bs4 import BeautifulSoup

url = 'https://www.binance.com/it/trade/PEPE_USDT?_from=markets&type=spot'
r = requests.get(url)
soup=BeautifulSoup(r.text, 'html.parser')
price = soup.title.text
>>> price
'0.00000121 | PEPEUSDT | Binance Spot'
>>> price = price.split()[0]
>>> price
'0.00000121'
>>> price = float(price)
>>> price
1.21e-06
>>> fixed_price =  0.00000110
>>> price < fixed_price
False
codeweak and buran like this post
Reply
#5
It's worth mention there is API for Binance
https://www.binance.com/en/binance-api

incl. light python library to connect to the API
codeweak likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#6
I thank everyone for the help, as you know I started about 4 days ago. And I started from 4 lines of code you can see at the top. After a few hours of studying I managed to complete the program.
.1 The program takes a numeric value from the web page and stores it. Deletes some characters and cleans the data
.2 the program prints the first data (price) with the complete time on a line
.3 the program prints recursively after 30 seconds (can be changed) in the line below updates the data constantly and also calculates the percentage of
variation and
the time in which the data is taken.
.4 makes a calculation to understand if it changes by a certain amount of percentage (which can be changed) and if it changes more than a certain value it
activates an audible warning and writes the updated price with the percentage of change.
Always update the data in the same line as the prompt, keeping only the first numerical data in the top line to calculate if it varies and by how much it varies


# This Python file uses the following encoding: utf-8
import requests
import chardet
import decimal
import numpy
import atexit
import os
import datetime
import time
import colorama # importa il modulo colorama
from colorama import Fore , Back , Style # importa la classe Fore che contiene i codici dei colori
from bs4 import BeautifulSoup 
colorama.init() # inizializza il modulo colorama
url = "https://www.binance.com/it/trade/SNT_USDT?_from=markets&type=spot"
r = requests.get(url)
soup=BeautifulSoup(r.text,'html.parser')
stringa = soup.title.text
lista = stringa.split("|")
numero1 = lista[0]
numero1 = numero1.strip().strip("<title data-shuvi-head=\"true\">")
print(" Valore iniziale",format(float(numero1),".8f"),"alle",datetime.datetime.now().strftime("%H:%M:%S"))
a = numero1
a= float(numero1)
#print (type(a))
continua = True
while continua:
    import requests
    from bs4 import BeautifulSoup 
    import winsound
    import chardet
    import decimal
    import datetime 
    import atexit
    import os
    import time
    import numpy
    import colorama 
    from colorama import Fore , Back , Style # importa la classe Fore che contiene i codici dei colori
    colorama.init() 
    url = "https://www.binance.com/it/trade/SNT_USDT?_from=markets&type=spot"
    r = requests.get(url)
    soup=BeautifulSoup(r.text,'html.parser')
    stringa = soup.title.text
    lista = stringa.split("|")
    numero = lista[0]
    numero = numero.strip().strip("<title data-shuvi-head=\"true\">")
    initial_value = float(numero1)
    current_value = float(numero)
    percentage_change = (float(current_value - float(initial_value)) / float(initial_value)) * 100
    if percentage_change >= 6:
        winsound.Beep(2000,6000)
    print("{:>27.8f} alle {:>8s}".format(float(numero),datetime.datetime.now().strftime("%H:%M:%S ")) + "Prezzo aggiornato" + f" {percentage_change} %",end='\r',flush=True)
    a = numero
    a= float(numero)
    time.sleep(30)
    
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Problem with scraping the Title from a web page Wagner822 0 687 Jun-29-2022, 11:31 PM
Last Post: Wagner822
  OCR-Python from Multi TIFF to HOCR getting only Data from 1st Page of multiple TIFF JOE 0 2,164 Feb-18-2022, 03:18 PM
Last Post: JOE
  simple html page with update data korenron 3 2,679 Nov-15-2021, 09:31 AM
Last Post: jamesaarr
  Looking for data/info on a perticular data-proccesing problem. MvGulik 9 3,907 May-01-2021, 07:43 AM
Last Post: MvGulik
  Extract data from PDF page to Excel nathan_nz 1 2,715 Oct-29-2020, 08:04 PM
Last Post: Larz60+
  Problem with serial data. How do I ignor incorrect data? donmerch 1 2,938 Jul-11-2017, 01:55 AM
Last Post: donmerch

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020