Is it possible to attach to an open existing Browser?

[tommytx]

I have for years been an avid script programmer using a program called Auto-It and I am sure many of you are very familiar with it... I love Python but I miss one of the most fantastic tools on the market.
That is using Auto-It to attach to a browser that has been opened manually..
For example you just give it the command to "Attach to = iE() or FF() and any commands that can be used on a browser that was opened by program can be applied to the attached IE or Firefox... super tool..
It should not be too hard to program for that.. I would certainly be open to offers to pay someone to figure out how to do it... Thanks.. Just wondering if anyone has seen such a tool floating out there...

[Larz60+]

FYI --
It may be easy, but I would expect not.
Nothing is difficult or impossible to the one who doesn't have to make it happen.

[tommytx]

I wish I was a super C programmer or something that could tie it in... The value is unbelievable. Its allows you to load page into the browser then attach to that particular browser and then control the browser with anything you like without loading that page over and over again... keeps you from getting banned when setting up a scraper..
Instead of hitting that same site a jillion times.. you load the site once and keep testing all the get this and get that. Sometimes I try to load the page and then save it to file and then use that saved htm page to load so I don't keep aggravating that same site over and over.. plus super fast since no need to load that page during hours of testing.
Not sure how hard it would be but if someone wanted to look at it and figure out how hard it will be I could easily spring for 300 to 500 bucks...depending on it it could then use the beautifly soup commands on the attached browser...anyone interested I am all ears...Thanks

[Larz60+]

Now this sounds more like selenium.
See: https://www.selenium.dev/
To learn how to use, run through the web scraping tutorials on this forum (I believe selenium is in part 2, but doesn't hurt to run all)
Web scraping part 1
Web scraping part 2

[tommytx]

Thanks for that suggestion Larz60+ but i have experimented with all of Pything that allows you to open and take control of a browser.. but it must be opened by your program. I am interested in taking control of a browser that is already open... with autoit you can take control of any opern browser firefox, IE, or chrome.. Which means that I can load the file manually then fire off my program and attach to that browser and control it... So I can mod my python program and re run it attaching only to that file.. That means during testing I load the file once not a jillion times... meanwhile the old browser lay lazily in minimize and i can come back and control it for days if i never close it..
So during testing I can load a facebook page which is a jillion bytes.. and never load it more than once a day and run grab classes, ids and all else and never load during testing...
With autoit i can have a dozen browsers open and minimized and call attach by title = "Warthogs in Columbia" and then switch to any of the other browsers the same way it searches open browsers for the title of the page that is displayed...and every command in the autoit arsenal can be played on the browser you are attached to... hope all that makes sense.
IF you hit facebook for the same page maybe 100 times in 15 min.. they will most likely be monitoring that ..

[tommytx]

Looks like this back in 2016 was a highly requested NEW feature.. but sorta got bogged down as too hard.. I found one on git hub that claims to work developed back in 2015 but I am not much on Git so not sure how to make it work..
Feature Request: OOPS! found the feature request URL but lost it now.. Looks like thousands were begging for this feature.. but they decided it was too hard..

Extensive program on github.com/antlong/selenium that claims to work.
If anyone can make heads or tails of this one at git i would appreciate it.. I may try to contact the developer but back in 2015 he may be hard to find.. but will hunt anyway..

[buran]

Why all the trouble to manually open browser and load a page to run tests on? It's more logical to save the page once and run the "test" completely offline? If that is the basis for the question you ask. At least run the "test" you refer to. Certainly there are "test" you need to perform against live page or mock of it. Also it looks like you mean not exactly tests, but more like experimenting.
And if we speak of real tests (i.e. with test suit), you can use setup and tear-down features/fixtures to prepare the environment for running multiple tests.

The problem is you try to replicate your workflow with AutoIt, when there are better alternatives.

[tommytx]

Thanks for the suggestion buran, but being a novice I have a problem downloading a page (any page) from FACEBOOK and then be able to load the saved page and use get this and get that.. and have it work. Many simple sites will let me do that but not Facebook and I am sure many of the other complex sites that fight against scrapers so they are so devious when you download a page in text or even in binary mode.. just will not work.. however as long as the browser is live i can do facebook calls all day long on the date inside the browser...

Course with your long time experience you may know how to download a page from facebook to a text file or html file and reload it later and be able to call all the same scraping commands... Works fine (save and reload) on the simpler pages but not one that is designed to give a scraper a hard time.

I am all ears if you have a few lines of Python script that will save a facebook page and then load from hard drive and do the same magic that can be done on a live page... if I could do that then i would not need any of this other stuff... and I no longer use AutoIt but many folks do and someone who does use it would know exactly what I am talking about with attach to a live browser..

Looking at what is in google...in the 2015 area many many folks want an attach capability and one person did it on git.. but being a novice I was not able to figure it out... I have some feelers out for and have sent him a message offering to pay him to give me enough detail to make it work for me...

But thanks for you ideas, but none of my attemts to save and reload ANY facebook page has worked...

I do understand that on the saved htm file no images or much other stuff will work due to the ref URL is gone but for scraping the saved data that won't matter.. Many times when I want the page to show images I can simply add <base href = url of the site it came from > but that is not needed for scraping the saved file...

<quote> It's more logical to save the page once and run the "test" completely offline?</quot>
Can I assume that you have done this before and can give me a few lines of code that will download any FACEBOOK page to text or html and allow me to load and continue scraping the saved file just like it is like.. if so I will forever be in your debt... Thank you.