Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Catch all cookies from any website
#1
Hey,

first of all sorry for my bad english but i trials it.

I try to create a cookie scanner that get all cookies (first and third party) from a website.
My first idea was to open a website with the selenium chromedriver and read the sqlite3 database which the chrome browser creates. I would do that because selenium itself can't read third party cookies. My problem is that the database is sometimes empty and sometimes not. If i open the database with python i get actually always an empty result but sometimes i open the database after my code is finished with a sqlite browser i get the cookies. I don't know why...

For example i open a mozilla website.

from selenium import webdriver
import os, shutil, sqlite3


browser_list_place = 0
browser_list = []
profiles_folder = "profiles"


def getcookies(url):
    if os.path.isdir(profiles_folder):
        shutil.rmtree(profiles_folder)

    co = webdriver.ChromeOptions()
    co.add_argument("--no-sandbox")
    co.add_argument("--user-data-dir=" + profiles_folder + "/" + str(browser_list_place))

    browser_list.append(webdriver.Chrome('D:\crawler\chromedriver.exe', options=co))
    browser_list[browser_list_place].set_page_load_timeout(30)

    browser_list[browser_list_place].get(url)

    #browser_list[browser_list_place].quit()

    for folder in range(0, browser_list_place + 1):
        con = sqlite3.connect(profiles_folder + "/" + str(folder) + "/Default/Cookies")
        cur = con.cursor()
        cur.execute("SELECT * FROM cookies")
        rows = cur.fetchall()
        for row in rows:
            print(row)


getcookies('https://developer.mozilla.org/de/')
In the chrome browser i can see 7 cookies.
[Image: Aq4p6.png]


Now after much problems i asking me if there a more efficient way to get all cookies from a website (first and third party) without selenium?

Thanks for your help!
Reply


Messages In This Thread
Catch all cookies from any website - by Chuky - Aug-14-2019, 01:14 PM
RE: Catch all cookies from any website - by Chuky - Aug-18-2019, 09:15 AM
RE: Catch all cookies from any website - by wavic - Aug-18-2019, 10:40 AM
RE: Catch all cookies from any website - by Chuky - Aug-18-2019, 12:11 PM
RE: Catch all cookies from any website - by Chuky - Aug-19-2019, 08:50 AM
RE: Catch all cookies from any website - by Chuky - Aug-20-2019, 10:56 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Saving/editing cookies? rhubarbpieguy 0 919 Dec-10-2022, 01:58 PM
Last Post: rhubarbpieguy
  Python cookies vj78 2 3,125 Apr-07-2021, 09:38 AM
Last Post: Naheed
  python scraping all cookies halberdd 1 4,860 Feb-26-2019, 12:10 PM
Last Post: halberdd
  Flask: Cookies for Saving User Input ? jomonetta 2 3,531 Nov-03-2018, 10:47 AM
Last Post: j.crater
  Inserting multiple cookies using python requests freshtomatoes 0 5,345 Aug-16-2018, 09:25 PM
Last Post: freshtomatoes
  Unable to load cookies with Pickle HiImNew 5 11,869 Jan-04-2018, 12:36 PM
Last Post: snippsat
  Missing cookies in session with requests sechot 0 6,292 Oct-22-2017, 12:09 PM
Last Post: sechot

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020