Python Forum
How to get html information from a tab of my default browser opened with webbrowser?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to get html information from a tab of my default browser opened with webbrowser?
#1
Question 
I have tried to figure out how can I get html information (such as h1 tags) from a new tab in my default browser that was opened with the following program:

    import tkinter as tk 
    #from tkinter import filedialog
    import webbrowser
    from bs4 import BeautifulSoup
    
    
    root = tk.Tk() #create a GUI element
    root.geometry('500x400') #resolution
    root.title("Bulkdozer") #Name of this program
    root.attributes('-topmost', True) #keep the program's window top-most
    
    def open_chrome_profile():
        soup = BeautifulSoup(webbrowser.open_new_tab('https://opensea.io/asset/create'), 'html.parser') #open a new tab using user's default browser
        content = soup.find('h1', {"class": "Blockreact__Block-sc-1xf18x6-0 Textreact__Text-sc-1w94ul3-0 dBFmez llcnwK"})
        print(content)
    
    #####BUTTON ZONE#######
    open_browser = tk.Button(root, width=20,  text="Open OpenSea Tab", command=open_chrome_profile) #executes the function when clicked
    open_browser.grid(row=22, column=1)
    #####BUTTON ZONE END#######
    root.mainloop()
The program above does open the corresponding tab, but also throws the following error at the line of soup variable:

Output:
TypeError: object of type 'bool' has no len()
And it doesn't print the content variable.

The reason for developing this in such a particular way is because I'm trying to make a program that will take control of user's default browser to make some repetitive tasks, and such automated process must be done using the default browser because it will have the Metamask extension that has been previously installed by the user.

May I know any ways of solving this problem?
Reply
#2
webbrowser.open_new_tab() returns boolean value. It does not return html source of the page you open.

You can use requests package or if the webpage uses JavaScript you should use tools like Selenium. webbrowser module from standard library is useless for a task such this.
Alternatively, or I would say as a better approach, you can explore the available third-party package to deal with this specific site/API
https://pypi.org/search/?q=opensea
noahverner1995 likes this post
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
(Jan-14-2022, 09:09 AM)buran Wrote: webbrowser.open_new_tab() returns boolean value. It does not return html source of the page you open.

You can use requests package or if the webpage uses JavaScript you should use tools like Selenium. webbrowser module from standard library is useless for a task such this.
Alternatively, or I would say as a better approach, you can explore the available third-party package to deal with this specific site/API
https://pypi.org/search/?q=opensea

Thank for the feedback, I think I managed to deal with it, here:

import tkinter as tk 
#from tkinter import filedialog
#import webbrowser
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service

root = tk.Tk()
root.geometry('500x400') #resolution
root.title("Bulkdozer") #Name of this program
root.attributes('-topmost', True) #keep the program's window top-most

opt = Options() #selenium options
opt.add_argument("--user-data-dir="+r"C:\Users\ResetStoreX\AppData\Local\Google\Chrome\User Data") #PATH profile
opt.add_argument('--profile-directory=Default') #Profile to use
s = Service('C:/Users/ResetStoreX/AppData/Local/Programs/Python/Python39/Scripts/chromedriver.exe')

def open_chrome_profile():
    #webbrowser.open_new_tab('https://opensea.io/asset/create') #open a new tab using user's default browser
    driver = webdriver.Chrome(service=s, options=opt) 
    driver.get('https://opensea.io/asset/create')
    
#####BUTTON ZONE#######
open_browser = tk.Button(root, width=20,  text="Open OpenSea Tab", command=open_chrome_profile) #executes the function when clicked
open_browser.grid(row=22, column=1)
#####BUTTON ZONE END#######
root.mainloop()
I removed the Default word from the profile PATH.

And I passed the absolute path of the ChromeDriver within the Service() function.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  does not save in other path than opened files before icode 3 906 Jun-23-2023, 07:25 PM
Last Post: snippsat
  Webbrowser.open() causes errors ChrisOfBristol 1 879 Apr-09-2023, 08:34 PM
Last Post: deanhystad
  Tkinterweb (Browser Module) Appending/Adding Additional HTML to a HTML Table Row AaronCatolico1 0 930 Dec-25-2022, 06:28 PM
Last Post: AaronCatolico1
  reading html and edit chekcbox to html jacklee26 5 3,078 Jul-01-2021, 10:31 AM
Last Post: snippsat
  Rmarkdown opened by python code - errors Rav013 0 2,092 Apr-27-2021, 03:13 PM
Last Post: Rav013
  How to get a URL from python 'webbrowser'? dheeraj 0 1,858 Apr-05-2021, 03:55 PM
Last Post: dheeraj
  Webbrowser jbrick97 3 59,010 Sep-27-2020, 03:58 AM
Last Post: ndc85430
  Making WebBrowser In PySide2 Harshil 0 1,900 Sep-16-2020, 05:03 PM
Last Post: Harshil
  webbrowser and menu not working. sik 1 1,833 Oct-31-2019, 03:39 AM
Last Post: newbieAuggie2019
  HTML to Python to Windows .bat and back to HTML perfectservice33 0 1,946 Aug-22-2019, 06:31 AM
Last Post: perfectservice33

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020