Python Forum
Download all secret links from a map design website
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Download all secret links from a map design website
#1
There is a website which shows links on a map (map layer currently can't be shown but links can be shown as points).
To view this website, this must be followed: (Pictures 1-2-3-4 also show the way)
Firstly, click this website 'http://svtbilgi.dsi.gov.tr/Sorgu.aspx', Picture 1
Secondly, choose '15. Kizilirmak Havzasi' from 'Havza' tab, Picture 2
Finally, click 'sorgula' bottom. Picture 3
After the final stage, you should view the website ('http://svtbilgi.dsi.gov.tr/HaritaNew.aspx') where the points can be shown on a map. Picture 4
Normally, I can use selenium to download webpages or can grab all links using different libraries. However, these methods can't obtain the links because they are embedded almost in a secret way.
I would like to download all links that these points have.
How can I get them?
from selenium import webdriver
from selenium.webdriver.support.ui import Select
from selenium.webdriver.support.ui import WebDriverWait


driver = webdriver.Firefox(executable_path=r'D:\geckodriver.exe')
driver.get("http://svtbilgi.dsi.gov.tr/Sorgu.aspx")
driver.find_element_by_id("ctl00_hld1_cbHavza").click()
Select(driver.find_element_by_id("ctl00_hld1_cbHavza")).select_by_visible_text("15. Kizilirmak Havzasi")
driver.find_element_by_id("ctl00_hld1_cbHavza").click()
driver.find_element_by_id("ctl00_hld1_btnListele").click()
parent_handle = driver.current_window_handle
all_urls = []
all_images = driver.find_elements_by_xpath("//div[contains(@id,'OL_Icon')]/img")
for image in all_images :
     image.click()
     for handle in driver.window_handles :
          if handle != parent_handle:
              driver.switch_to_window(handle)
              WebDriverWait(driver, 5).until(lambda d: d.execute_script('return document.readyState') == 'complete')
              all_urls.append(driver.current_url)
              driver.close()
              driver.switchTo.window(parent_handle 
This script doesn't continue after 'parent_handle = driver.current_window_handle' line.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  All product links to products on a website MarionStorm 0 1,057 Jun-02-2022, 11:17 PM
Last Post: MarionStorm
  Website scrapping and download santoshrane 3 4,257 Apr-14-2021, 07:22 AM
Last Post: kashcode
  Login and download an exported csv file within a ribbon/button in a website Alekhya 0 2,617 Feb-26-2021, 04:15 PM
Last Post: Alekhya
  Cant Download Images from Unsplash Website firaki12345 1 2,258 Feb-08-2021, 04:15 PM
Last Post: buran
  Extracting all the links on a website randeniyamohan 1 4,503 Jan-09-2020, 04:47 PM
Last Post: Clunk_Head
  Python Rest API Public/Secret Keys Authentication Nonce dn237 1 2,873 Oct-31-2019, 02:07 AM
Last Post: dn237
  Extracting links from website with selenium bs4 and python M1ck0 1 3,690 Jul-20-2019, 10:29 PM
Last Post: Larz60+
  download pdf file from website m_annur2001 1 2,956 Jun-21-2019, 05:03 AM
Last Post: j.crater
  webscrapping links and then enter those links to scrape data kirito85 2 3,147 Jun-13-2019, 02:23 AM
Last Post: kirito85
  I wan't to Download all .zip Files From A Website (Project AI) eddywinch82 68 37,469 Oct-28-2018, 02:13 PM
Last Post: eddywinch82

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020