Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Web Scrapping Through API
#1
Hi team, Im new to python/development. As i started to read the requests module, i came across this weird website terion.in. Its a game website were the user has to guess the next sequence of colors. These color generate randomly based on the numbers. new set appears in 3 min of time gap. when i checked out the API, it was in Get method and i was able to scrape the data.
import requests

headers = {
    'authority': 'booe.in',
    'accept': 'application/json, text/plain, */*',
    'authorization': 'Token bee0043a8dd3413efa50e91853c6e9b1ca819a7e',
    'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36',
    'sec-fetch-site': 'same-origin',
    'sec-fetch-mode': 'cors',
    'sec-fetch-dest': 'empty',
    'referer': 'https://booe.in/',
    'accept-language': 'en-US,en;q=0.9,mt;q=0.8',
    'cookie': '__cfduid=d6548db2ec1334ee4806517500f0054fc1596897016; coem.notice.today=Important^%^20note^%^3A^%^20The^%^20recharge^%^20channel^%^20and^%^20withdrawal^%^20channel^%^20are^%^20maintained^%^20and^%^20upgraded.^%^20thank^%^20you^%^20for^%^20your^%^20support^!',
    
}

params = (
    ('category', 'P^'),
    ('p', '2^'),
    ('p_size', '2'),    
)
response = requests.get('https://booe.in/win/guesses', headers=headers, params=params)
json_response = response.json() 
json_response 
output is like

{'code': 200,
'page': {'count': 44136, 'num_pages': 22068, 'current_page': 1},
'queryset': [{'period': 20200811455,
'price': 26324,
'last_num': 4,
'is_green': False,
'is_red': True,
'is_violet': False,
'create_time': '2020-08-11T22:45:02.736176+05:30'},
{'period': 20200811454,
'price': 26321,
'last_num': 1,
'is_green': True,
'is_red': False,
'is_violet': False,
'create_time': '2020-08-11T22:42:02.844054+05:30'}]}

So in this the next sequence generation will be '20200811456','20200811457',.....and so on.
the question is can i request a specific key - 'period number' and ask for value - 'result'?
Reply
#2
the site terion.in redirects to some online shop. you should check the API docs for info.

on a second read - I guess you don't know what API means. you just replicate the request made by the site (they may make request to some internal API, but it's not official, exposed to public). I guess they just send the current status, and you cannot make forward- or backwards- looking requests
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
Thanks buran,

Im just beginner, thanks for ur patience
The site is simply a dummy shopping website, if you login, it will add up a game page. in which requests are made and actual process of that game reveals only after the login.

neglect if my question irrelevant.
Reply
#4
my point is - without official API docs our guess is best as yours. one can try to reveal some of the endpoints of an internal API if they can replicate the requests via website and explore the requests made in browser.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Problem with scrapping Website giddyhead 1 1,580 Mar-08-2024, 08:20 AM
Last Post: AhanaSharma
  python web scrapping mg24 1 269 Mar-01-2024, 09:48 PM
Last Post: snippsat
  How can I ignore empty fields when scrapping never5000 0 1,353 Feb-11-2022, 09:19 AM
Last Post: never5000
  Suggestion request for scrapping html table Vkkindia 3 1,986 Dec-06-2021, 06:09 PM
Last Post: Larz60+
  web scrapping through Python Naheed 2 2,576 May-17-2021, 12:02 PM
Last Post: Naheed
  Website scrapping and download santoshrane 3 4,257 Apr-14-2021, 07:22 AM
Last Post: kashcode
  Newbie help with lxml scrapping chelsealoa 1 1,827 Jan-08-2021, 09:14 AM
Last Post: Larz60+
  Scrapping Sport score laplacea 1 2,220 Dec-13-2020, 04:09 PM
Last Post: Larz60+
  How to export to csv the output of every iteration when scrapping with a loop efthymios 2 2,240 Nov-30-2020, 07:46 PM
Last Post: efthymios
  Web scrapping - Stopped working peterjv26 2 3,009 Sep-23-2020, 08:30 AM
Last Post: peterjv26

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020