Python Forum
python how to check short url working?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
python how to check short url working?
#1
hi all,
i have problem with test short url working or not. short url redirect two times or more to item page.
http://jd.cn.hn/aeXb --> https://u.jd.com/mecUmb ---> https://jingfen.jd.com/item.html?sku=658...978a383159

import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
for resp in response.history:
    print(resp.status_code, resp.url)
print(response.url)
301 http://jd.cn.hn/aeXb
302 https://www.weibo.com/
https://passport.weibo.com/visitor/visit...77202.8249
i can't find the right way for get final page. please advice. thanks.
Reply
#2
the last url in response.history is the same as response.url after all redirects:
import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)
Output:
['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas'] https://www.weibo.com/overseas
if I run your code, I get the same:
Output:
301 http://jd.cn.hn/aeXb 302 https://www.weibo.com/ 301 http://weibo.com/overseas https://www.weibo.com/overseas
I see just one difference between response.history and response.url - the schema http vs https
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
(Mar-16-2020, 05:02 PM)buran Wrote: the last url in response.history is the same as response.url after all redirects:
import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)
Output:
['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas'] https://www.weibo.com/overseas
if I run your code, I get the same:
Output:
301 http://jd.cn.hn/aeXb 302 https://www.weibo.com/ 301 http://weibo.com/overseas https://www.weibo.com/overseas
I see just one difference between response.history and response.url - the schema http vs https

thanks for reply. that's why i ask here. i think we get link with wrong way. here is how it work in browser
imgur.com/a/2EyQDyK
Reply
#4
For me url in the browser and via requests are the same
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#5
(Mar-17-2020, 04:52 AM)buran Wrote: For me url in the browser and via requests are the same

i think you have try more that they have blocked you for a while.
Reply
#6
No, I tried just once.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#7
(Mar-17-2020, 06:33 AM)buran Wrote: No, I tried just once.

then i don't know why.
i try to post all redirect link here in order.
Output:
https://u.jd.com/mecUmb https://u.jd.com/jda?e=99_2|1_10_1|||&p=AyIGZRhaFQEaBFcaWBQyFQNcHVoSARQFVBlrUV1KWQorAlBHU0VeBUVOWk1RAk8ECllHGAdFBwtaV1MJBAJQXk8JF0EfGQUWDlMaXBYEEAZXDBsZdRBjBm8SclxmBQd7LGZpRXMdYzh1W1QONhk7fGJ2b1xsMmJ1a2M2WzhsYWVnIWgeYl1EYDVCDlV2EH8NbAIVQGV%2BIUsvTFdFYB18JHV2YkUCTTBecRNwFnsTcVJmdD0eOxdicHM2b08WdWVYKUgvYXVXcTx4K3V2R0IhcgFxYmEHEmMiYWVmXj5aL3x2cXMde08XdyJAV0wQdhxBcTASH0xkF3oQHQllZlR3WRdrFDIRBlUbWR0AFARVK2sVBiJGOxtaFwMXAlcSaxQyEgNcHlkTBRUPVxNdFTISAWXP05vWlKiCntHBubjQ8ZqMqZ3Ft8bPzqkyIjdWK1MdbBFKVXVbFGwSShhWayUCETdWKwRRX083VhpaFAI%3D&a=fCg9UgoiAwwHO1BcXkQYFFlgfnxweFFeSF0zVRBSUll%2bAQAPDSwjLw%3d%3d&refer=norefer&d=mecUmb https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa
last link is what i want to get with python.
Reply
#8
looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#9
(Mar-17-2020, 06:53 AM)buran Wrote: looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?

i don't use proxy for that. it may get your location and others info.
Reply
#10
then I don't have an idea. for me the browser and code produce the same result. maybe someone else would be able to shed light.
I will move this thread to webscrapping, because it looks more appropriate sub-forum.
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Short link URL Evil_Patrick 5 3,093 Jul-10-2020, 10:14 AM
Last Post: steve_shambles
  Simple Element Check Code in Selenium Not Working digitalmatic7 1 3,009 Feb-18-2018, 06:53 AM
Last Post: metulburr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020