Dec-29-2016, 07:03 AM
is urllib2 a good choice for getting website contents? it is used in that code i posted regarding catchin IOError failing. it gets my website OK as well as google. but when i append the search stuff it fails on both sites (while lynx and curl work). i cross checked what urllib2 and lynx sent to a dummy local website and they both look right.
more testing:
note that my website does not have /search script so i just used the apex page for these tests.
more testing:
note that my website does not have /search script so i just used the apex page for these tests.
Output:lt1/forums /home/forums 1> python filetopystr.py 0 'http://www.google.com/' ip 80|wc -c
connecting to 'http://www.google.com/'
connected to 'http://www.google.com/'
reading from 'http://www.google.com/'
read 10872 bytes
11478
lt1/forums /home/forums 2> python filetopystr.py 0 'http://linuxhomepage.com/' ip 80|wc -c
connecting to 'http://linuxhomepage.com/'
connected to 'http://linuxhomepage.com/'
reading from 'http://linuxhomepage.com/'
read 30994 bytes
32900
lt1/forums /home/forums 3> python filetopystr.py 0 'http://ipv6.linuxhomepage.com/' ip 80|wc -c
connecting to 'http://ipv6.linuxhomepage.com/'
connected to 'http://ipv6.linuxhomepage.com/'
reading from 'http://ipv6.linuxhomepage.com/'
read 30988 bytes
32894
lt1/forums /home/forums 4> python filetopystr.py 0 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c
connecting to 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address'
Error connecting to 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address'
0
lt1/forums /home/forums 5> python filetopystr.py 0 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c
connecting to 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
connected to 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
reading from 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
read 30994 bytes
32900
lt1/forums /home/forums 6> python filetopystr.py 0 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c
connecting to 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
connected to 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
reading from 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'
read 30988 bytes
32894
lt1/forums /home/forums 7> lynx -mime_header 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address'|wc -c
19816
lt1/forums /home/forums 8> lynx -mime_header 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'|wc -c
29505
lt1/forums /home/forums 9> lynx -mime_header 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'|wc -c
29499
lt1/forums /home/forums 10>
Tradition is peer pressure from dead people
What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.