Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 page impossible to scrap? :O
Hi guys,

i have found page where i cant scrap the shortest flight

how it come? it is possible to block site from scrapping?

page = ''
r = requests.get(page)
content = (r.text)
soup = BeautifulSoup(content, 'html.parser')
test = soup.find_all(class_='BpkTicket_bpk-ticket__paper__2gPSe BpkTicket_bpk-ticket__main__J31fH BpkTicket_bpk-ticket__main--padded__WIbjx BpkTicket_bpk-ticket__main--horizontal__2MgwA BpkTicket_bpk-ticket__paper--with-notches__19yQc'):
I guess flight seeker sites works with some kind of refresh data, hence, its not visible in requests? am i right? In this case i would need some sleep/wait function, right?
If they use JavaScript you may need to use Selenium
Check our tutorial -
look for God dammit JavaScript, why do i not get all content and next
zarize likes this post
Thanks buran,

as always helpful! :)

now time to learn captcha solving! :D

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  use Xpath in Python :: libxml2 for a page-to-page skip-setting apollo 2 372 Mar-19-2020, 06:13 PM
Last Post: apollo
  Scrap a dynamic span hefaz 0 577 Mar-07-2020, 02:56 PM
Last Post: hefaz
  scrap by defining 3 functions zarize 0 260 Feb-18-2020, 03:55 PM
Last Post: zarize
  Skipping anti-scrap zarize 0 309 Jan-17-2020, 11:51 AM
Last Post: zarize
  Cannot get selenium to scrap past the first two pages newbie_programmer 0 959 Dec-12-2019, 06:19 AM
Last Post: newbie_programmer
  Scrap data from not standarized page? zarize 4 600 Nov-25-2019, 10:25 AM
Last Post: zarize
  Scrap a value from website harsush 1 379 Aug-29-2019, 01:57 PM
Last Post: snippsat
  Scrap text out of td table from URLS Gochix2020 4 1,267 Aug-03-2019, 02:56 AM
Last Post: Larz60+
  scrap macrotrends mr_byte31 7 2,017 Aug-02-2019, 12:02 AM
Last Post: mr_byte31
  Scrap arbitrage odds -help Gochix2020 3 637 Jul-31-2019, 10:45 AM
Last Post: Gochix2020

Forum Jump:

Users browsing this thread: 1 Guest(s)