Python Forum

Full Version: python how to check short url working?
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
hi all,
i have problem with test short url working or not. short url redirect two times or more to item page.
http://jd.cn.hn/aeXb --> https://u.jd.com/mecUmb ---> https://jingfen.jd.com/item.html?sku=658...978a383159

import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
for resp in response.history:
    print(resp.status_code, resp.url)
print(response.url)
301 http://jd.cn.hn/aeXb
302 https://www.weibo.com/
https://passport.weibo.com/visitor/visit...77202.8249
i can't find the right way for get final page. please advice. thanks.
the last url in response.history is the same as response.url after all redirects:
import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)
Output:
['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas'] https://www.weibo.com/overseas
if I run your code, I get the same:
Output:
301 http://jd.cn.hn/aeXb 302 https://www.weibo.com/ 301 http://weibo.com/overseas https://www.weibo.com/overseas
I see just one difference between response.history and response.url - the schema http vs https
(Mar-16-2020, 05:02 PM)buran Wrote: [ -> ]the last url in response.history is the same as response.url after all redirects:
import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)
Output:
['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas'] https://www.weibo.com/overseas
if I run your code, I get the same:
Output:
301 http://jd.cn.hn/aeXb 302 https://www.weibo.com/ 301 http://weibo.com/overseas https://www.weibo.com/overseas
I see just one difference between response.history and response.url - the schema http vs https

thanks for reply. that's why i ask here. i think we get link with wrong way. here is how it work in browser
imgur.com/a/2EyQDyK
For me url in the browser and via requests are the same
(Mar-17-2020, 04:52 AM)buran Wrote: [ -> ]For me url in the browser and via requests are the same

i think you have try more that they have blocked you for a while.
No, I tried just once.
(Mar-17-2020, 06:33 AM)buran Wrote: [ -> ]No, I tried just once.

then i don't know why.
i try to post all redirect link here in order.
Output:
https://u.jd.com/mecUmb https://u.jd.com/jda?e=99_2|1_10_1|||&p=AyIGZRhaFQEaBFcaWBQyFQNcHVoSARQFVBlrUV1KWQorAlBHU0VeBUVOWk1RAk8ECllHGAdFBwtaV1MJBAJQXk8JF0EfGQUWDlMaXBYEEAZXDBsZdRBjBm8SclxmBQd7LGZpRXMdYzh1W1QONhk7fGJ2b1xsMmJ1a2M2WzhsYWVnIWgeYl1EYDVCDlV2EH8NbAIVQGV%2BIUsvTFdFYB18JHV2YkUCTTBecRNwFnsTcVJmdD0eOxdicHM2b08WdWVYKUgvYXVXcTx4K3V2R0IhcgFxYmEHEmMiYWVmXj5aL3x2cXMde08XdyJAV0wQdhxBcTASH0xkF3oQHQllZlR3WRdrFDIRBlUbWR0AFARVK2sVBiJGOxtaFwMXAlcSaxQyEgNcHlkTBRUPVxNdFTISAWXP05vWlKiCntHBubjQ8ZqMqZ3Ft8bPzqkyIjdWK1MdbBFKVXVbFGwSShhWayUCETdWKwRRX083VhpaFAI%3D&a=fCg9UgoiAwwHO1BcXkQYFFlgfnxweFFeSF0zVRBSUll%2bAQAPDSwjLw%3d%3d&refer=norefer&d=mecUmb https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa
last link is what i want to get with python.
looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?
(Mar-17-2020, 06:53 AM)buran Wrote: [ -> ]looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?

i don't use proxy for that. it may get your location and others info.
then I don't have an idea. for me the browser and code produce the same result. maybe someone else would be able to shed light.
I will move this thread to webscrapping, because it looks more appropriate sub-forum.
Pages: 1 2