Web scraping (selenium (i think))

[metulburr]

Here seems to be a method (responses 2 and 3) for win and linux
https://stackoverflow.com/questions/1096...n-selenium

The linux one works for me with selenium to save as in firefox with a few modifications

sudo apt-get install xautomation

from subprocess import Popen, PIPE

save_sequence = b"""keydown Control_L
key S
keyup Control_L
keydown Return
"""


def keypress(sequence):
    p = Popen(['xte'], stdin=PIPE)
    p.communicate(input=sequence)

keypress(save_sequence)

it downloads the css, js, images in a directory

metulburr@ubuntu:~/Downloads$ cd Twitter_files/
metulburr@ubuntu:~/Downloads/Twitter_files$ ls
0.commons.en.e39ce78c2d3da.js     saved_resource(5).html
8.pages_home.en.01b37cfab9b.js  saved_resource(6).html
analytics.js                             saved_resource(7).html
default_profile_bigger.png               saved_resource(8).html
default_profile_normal.png               saved_resource(9).html
delight_prompt.png                       saved_resource.html
init.en.244fa41b6a57.js          twitter_core.bundle.css
Qtz8LqPx_bigger.jpg                      twitter_more_1.bundle.css
saved_resource(1).html                   twitter_more_2.bundle.css
saved_resource(2).html                   twitter_profile_editing.bundle.css
saved_resource(3).html                   Vb8k670S_bigger.png
saved_resource(4).html                   y-kyowAV_bigger.jpg
metulburr@ubuntu:~/Downloads/Twitter_files$