python how to check short url working?

python how to check short url working? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html)
+--- Thread: python how to check short url working? (/thread-25035.html)

Pages: 1 2

python how to check short url working? - Pyguys - Mar-16-2020

hi all,
i have problem with test short url working or not. short url redirect two times or more to item page.
http://jd.cn.hn/aeXb --> https://u.jd.com/mecUmb ---> https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=ebbb02b79a614899bd93b3978a383159

import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
for resp in response.history:
    print(resp.status_code, resp.url)
print(response.url)

301 http://jd.cn.hn/aeXb
302 https://www.weibo.com/
https://passport.weibo.com/visitor/visitor?entry=miniblog&a=enter&url=https%3A%2F%2Fweibo.com%2F&domain=.weibo.com&ua=php-sso_sdk_client-0.6.28&_rand=1584377202.8249
i can't find the right way for get final page. please advice. thanks.

RE: python how to check short url working? - buran - Mar-16-2020

the last url in response.history is the same as response.url after all redirects:

import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)

Output:['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas']
https://www.weibo.com/overseas

if I run your code, I get the same:

Output:301 http://jd.cn.hn/aeXb
302 https://www.weibo.com/
301 http://weibo.com/overseas
https://www.weibo.com/overseas

I see just one difference between response.history and response.url - the schema http vs https

RE: python how to check short url working? - Pyguys - Mar-16-2020

(Mar-16-2020, 05:02 PM)buran Wrote: the last url in response.history is the same as response.url after all redirects:
import requests
url = 'http://jd.cn.hn/aeXb'
response = requests.get(url)
print([resp.url for resp in response.history])
print(response.url)
Output:['http://jd.cn.hn/aeXb', 'https://www.weibo.com/', 'http://weibo.com/overseas']
https://www.weibo.com/overseas
if I run your code, I get the same:
Output:301 http://jd.cn.hn/aeXb
302 https://www.weibo.com/
301 http://weibo.com/overseas
https://www.weibo.com/overseas
I see just one difference between response.history and response.url - the schema http vs https

thanks for reply. that's why i ask here. i think we get link with wrong way. here is how it work in browser
imgur.com/a/2EyQDyK

RE: python how to check short url working? - buran - Mar-17-2020

For me url in the browser and via requests are the same

RE: python how to check short url working? - Pyguys - Mar-17-2020

(Mar-17-2020, 04:52 AM)buran Wrote: For me url in the browser and via requests are the same

i think you have try more that they have blocked you for a while.

RE: python how to check short url working? - buran - Mar-17-2020

No, I tried just once.

RE: python how to check short url working? - Pyguys - Mar-17-2020

(Mar-17-2020, 06:33 AM)buran Wrote: No, I tried just once.

then i don't know why.
i try to post all redirect link here in order.

Output:https://u.jd.com/mecUmb

https://u.jd.com/jda?e=99_2|1_10_1|||&p=AyIGZRhaFQEaBFcaWBQyFQNcHVoSARQFVBlrUV1KWQorAlBHU0VeBUVOWk1RAk8ECllHGAdFBwtaV1MJBAJQXk8JF0EfGQUWDlMaXBYEEAZXDBsZdRBjBm8SclxmBQd7LGZpRXMdYzh1W1QONhk7fGJ2b1xsMmJ1a2M2WzhsYWVnIWgeYl1EYDVCDlV2EH8NbAIVQGV%2BIUsvTFdFYB18JHV2YkUCTTBecRNwFnsTcVJmdD0eOxdicHM2b08WdWVYKUgvYXVXcTx4K3V2R0IhcgFxYmEHEmMiYWVmXj5aL3x2cXMde08XdyJAV0wQdhxBcTASH0xkF3oQHQllZlR3WRdrFDIRBlUbWR0AFARVK2sVBiJGOxtaFwMXAlcSaxQyEgNcHlkTBRUPVxNdFTISAWXP05vWlKiCntHBubjQ8ZqMqZ3Ft8bPzqkyIjdWK1MdbBFKVXVbFGwSShhWayUCETdWKwRRX083VhpaFAI%3D&a=fCg9UgoiAwwHO1BcXkQYFFlgfnxweFFeSF0zVRBSUll%2bAQAPDSwjLw%3d%3d&refer=norefer&d=mecUmb

https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa



https://jingfen.jd.com/item.html?sku=65870627303&q=F3UbExVoE3cQFBZfEyIRQhw8R3QXQUY8FXFFHURqRHRFQEBtFngVQhdqE3IiFh1sFHEaEhdfVyVNQEAsfgZzB0FrQyUaEBY4Q3QSERE/FnMbEEFtGXRAQEdtEXkUQB1vIHEVEhZpEXEREyQ=&uabt=37_63&d=mecUmb&cu=true&utm_source=kong&utm_medium=jingfen&utm_campaign=t_2011393721_&utm_term=fad316ae5d004bc18fafc50dc39164aa

last link is what i want to get with python.

RE: python how to check short url working? - buran - Mar-17-2020

looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?

RE: python how to check short url working? - Pyguys - Mar-17-2020

(Mar-17-2020, 06:53 AM)buran Wrote: looking at the urls I get - is it possible that what I see is due to my geo location? url I see says overseas
Do you use some proxy in the browser to simulate a requests from a specific location?

i don't use proxy for that. it may get your location and others info.

RE: python how to check short url working? - buran - Mar-17-2020

then I don't have an idea. for me the browser and code produce the same result. maybe someone else would be able to shed light.
I will move this thread to webscrapping, because it looks more appropriate sub-forum.