Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
urlib2 issues
#1
is urllib2 a good choice for getting website contents? it is used in that code i posted regarding catchin IOError failing. it gets my website OK as well as google.  but when i append the search stuff it fails on both sites (while lynx and curl work).  i cross checked what urllib2 and lynx sent to a dummy local website and they both look right.

more testing:

note that my website does not have /search script so i just used the apex page for these tests.

Output:
lt1/forums /home/forums 1> python filetopystr.py 0 'http://www.google.com/' ip 80|wc -c connecting to 'http://www.google.com/' connected  to 'http://www.google.com/' reading  from 'http://www.google.com/' read 10872 bytes 11478 lt1/forums /home/forums 2> python filetopystr.py 0 'http://linuxhomepage.com/' ip 80|wc -c connecting to 'http://linuxhomepage.com/' connected  to 'http://linuxhomepage.com/' reading  from 'http://linuxhomepage.com/' read 30994 bytes 32900 lt1/forums /home/forums 3> python filetopystr.py 0 'http://ipv6.linuxhomepage.com/' ip 80|wc -c connecting to 'http://ipv6.linuxhomepage.com/' connected  to 'http://ipv6.linuxhomepage.com/' reading  from 'http://ipv6.linuxhomepage.com/' read 30988 bytes 32894 lt1/forums /home/forums 4> python filetopystr.py 0 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c connecting to 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address' Error connecting to 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address' 0 lt1/forums /home/forums 5> python filetopystr.py 0 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c connecting to 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' connected  to 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' reading  from 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' read 30994 bytes 32900 lt1/forums /home/forums 6> python filetopystr.py 0 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' ip 80|wc -c connecting to 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' connected  to 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' reading  from 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address' read 30988 bytes 32894 lt1/forums /home/forums 7> lynx -mime_header 'http://www.google.com/search?hl=en&as_q=what+is+my+ip+address'|wc -c 19816 lt1/forums /home/forums 8> lynx -mime_header 'http://linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'|wc -c 29505 lt1/forums /home/forums 9> lynx -mime_header 'http://ipv6.linuxhomepage.com/?hl=en&as_q=what+is+my+ip+address'|wc -c 29499 lt1/forums /home/forums 10>
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Messages In This Thread
urlib2 issues - by Skaperen - Dec-29-2016, 07:03 AM
RE: urlib2 issues - by Larz60+ - Dec-29-2016, 07:19 AM
RE: urlib2 issues - by snippsat - Dec-29-2016, 07:38 AM
RE: urlib2 issues - by Skaperen - Dec-29-2016, 07:44 AM
RE: urlib2 issues - by snippsat - Dec-29-2016, 08:03 AM
RE: urlib2 issues - by Skaperen - Dec-29-2016, 08:53 AM
RE: urlib2 issues - by wavic - Dec-29-2016, 08:22 AM
RE: urlib2 issues - by wavic - Dec-29-2016, 09:04 AM
RE: urlib2 issues - by Skaperen - Dec-29-2016, 09:08 AM
RE: urlib2 issues - by wavic - Dec-29-2016, 09:30 AM
RE: urlib2 issues - by Skaperen - Dec-29-2016, 10:04 AM
RE: urlib2 issues - by wavic - Dec-29-2016, 10:15 AM
RE: urlib2 issues - by Skaperen - Dec-30-2016, 04:59 AM
RE: urlib2 issues - by wavic - Dec-30-2016, 07:53 AM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020