Python Forum

I want to loop a url for scrape all products info from each pages but two parameters of this url automatically changes when it go to next page. This two parameters changes "2&qid=1552585481" and "&ref=sr_pg_2" such as when I go to 3rd page it will be "&qid=1552585493' and "&ref=sr_pg_3". Below my code given:

import requests
from bs4 import BeautifulSoup

url = 'https://www.amazon.com/s?k=%2Fwomen-dress%2Fs%3Fk%3Dwomen+dress&qid=1552587323&ref=sr_pg_1'

#page_url_setup
for page in range(10): 
    print('---', page, '---')
    r = requests.get(url + str(page))
    soup = BeautifulSoup(r.content, "html.parser")
    for link in soup.find_all('a',{'class':'ebayui-pagination__control','rel':'next'}):
        print("<a href='>%s'>%s</a>" % (link.get("href"), link.text))

That number changes for each new access to any page.
You cannot override this (or I least I don't know how to override this).
You'll need to use selenium in order to process a page of this type.
Please look at the tutorials supplied on this forum.
https://python-forum.io/Thread-Web-Scraping-part-1
and
https://python-forum.io/Thread-Web-scraping-part-2