Python Forum
web crawler that retrieves data not stored in source code
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
web crawler that retrieves data not stored in source code
#3
sure, the website is publi24.ro
it doesn't really have an english version but what you're looking for is:

publi24.ro/anunturi/locuri-de-munca/bucuresti/
or with page increment
publi24.ro/anunturi/locuri-de-munca/bucuresti/?pag=2
that's the main page with ads from a section (i picked the job seeking section)

and an ad page would look like this:
publi24.ro/anunturi/locuri-de-munca/anunt/Echipa-Tehnician-Alpinist-Telecom/7b00667478616b51.html

if you inspect element here and hover over the view count number (in this case "1") you'll see a tag that says: <span add-view="18230886">1</span> (it's not 1 anymore cuz i refreshed a bunch of times but you get the idea) :)
but in the source code it doesn't show anything
i also run my program and it brings up None as well.

i'm thinking maybe if i could make python run the page as if it were a user instead of grabbing the source code maybe it would pull up the view number and print it but i'm not sure. :-/
Reply


Messages In This Thread
RE: web crawler that retrieves data not stored in source code - by edithegodfather - Jan-05-2017, 01:24 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Hide source code from python process itself xmghe 2 1,884 Jan-27-2021, 04:04 PM
Last Post: xmghe
  Web Crawler help Mr_Mafia 2 1,899 Apr-04-2020, 07:20 PM
Last Post: Mr_Mafia
  scraping from a website that hides source code PIWI_Protein 1 1,972 Mar-27-2020, 05:08 PM
Last Post: Larz60+
  Web Crawler help takaa 39 27,282 Apr-26-2019, 12:14 PM
Last Post: stateitreal
  Python requests.get() returns broken source code instead of expected source code? FatalPythonError 3 3,731 Sep-21-2018, 02:46 PM
Last Post: nilamo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020