Python Forum

Full Version: Web Scrapping Through API
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Hi team, Im new to python/development. As i started to read the requests module, i came across this weird website terion.in. Its a game website were the user has to guess the next sequence of colors. These color generate randomly based on the numbers. new set appears in 3 min of time gap. when i checked out the API, it was in Get method and i was able to scrape the data.
import requests

headers = {
    'authority': 'booe.in',
    'accept': 'application/json, text/plain, */*',
    'authorization': 'Token bee0043a8dd3413efa50e91853c6e9b1ca819a7e',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    'referer': 'https://booe.in/',
    'accept-language': 'en-US,en;q=0.9,mt;q=0.8',
    'cookie': '__cfduid=d6548db2ec1334ee4806517500f0054fc1596897016; coem.notice.today=Important^%^20note^%^3A^%^20The^%^20recharge^%^20channel^%^20and^%^20withdrawal^%^20channel^%^20are^%^20maintained^%^20and^%^20upgraded.^%^20thank^%^20you^%^20for^%^20your^%^20support^!',
    
}

params = (
    ('category', 'P^'),
    ('p', '2^'),
    ('p_size', '2'),    
)
response = requests.get('https://booe.in/win/guesses', headers=headers, params=params)
json_response = response.json() 
json_response 
output is like

{'code': 200,
'page': {'count': 44136, 'num_pages': 22068, 'current_page': 1},
'queryset': [{'period': 20200811455,
'price': 26324,
'last_num': 4,
'is_green': False,
'is_red': True,
'is_violet': False,
'create_time': '2020-08-11T22:45:02.736176+05:30'},
{'period': 20200811454,
'price': 26321,
'last_num': 1,
'is_green': True,
'is_red': False,
'is_violet': False,
'create_time': '2020-08-11T22:42:02.844054+05:30'}]}

So in this the next sequence generation will be '20200811456','20200811457',.....and so on.
the question is can i request a specific key - 'period number' and ask for value - 'result'?
the site terion.in redirects to some online shop. you should check the API docs for info.

on a second read - I guess you don't know what API means. you just replicate the request made by the site (they may make request to some internal API, but it's not official, exposed to public). I guess they just send the current status, and you cannot make forward- or backwards- looking requests
Thanks buran,

Im just beginner, thanks for ur patience
The site is simply a dummy shopping website, if you login, it will add up a game page. in which requests are made and actual process of that game reveals only after the login.

neglect if my question irrelevant.
my point is - without official API docs our guess is best as yours. one can try to reveal some of the endpoints of an internal API if they can replicate the requests via website and explore the requests made in browser.