Python Forum

Full Version: scraping multiple pages of a website.
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Pages: 1 2
Thank you all, I try your one first buran and it worked great. I am working on your now Larz60+, I hope both are you will not mind if I don't understand something that I can ask you to explain it to me. Thanks for all your help. I have a lot of work to now.
Hi Buran, I took you code apart peace by peace. I know every bit of it now, but for this.
Tell what this does how it works
url = '{}{}'.format(base_url, letter)
what does '{} {}' this do
I try to find .format in the requests Mod. but could not.
what is and what does .format do
I know what base_url is and I think letter is the letter at the end of the web page address. If you can, I need to know everything about this line of code.
Thank you
Renny
url = '{}{}'.format(base_url, letter) is a method used to concatenate strings, combining
"https://www.usa.gov/federal-agencies/" and "a" to create "https://www.usa.gov/federal-agencies/a"

For a discussion of various string concatenation methods, see: https://softwareengineering.stackexchang...catenation

The following code (taken from this thread) compares two different concatenation methods. The method used by Buran seems to be preferred for readability for all but the simplest of concatenations.
from string import ascii_lowercase

base_url  = 'https://www.usa.gov/federal-agencies/'
for letter in ascii_lowercase:
    url_one = '{}{}'.format(base_url, letter)
    url_two = base_url + letter
    print("{}  {}".format(letter, url_one))
    print("{}  {}".format(letter, url_two))    
Lewis
Lewis already explained, but here is the docs. Read about string-formating mini language

https://docs.python.org/3.6/library/stri...formatting

in 3.6+ one can use f-strings that Larz60+ used in his post#2 https://python-forum.io/Thread-scraping-...4#pid49344
Thank you all, nice Doc buran, I enjoy reading it. I have try it on other site and it worked fine but for one. I did get it to work on that one. Thank you all so much.
renny
Pages: 1 2