Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Scraping Issue with BS
#6
ok. I will give it a shot. Thanks.

(Dec-08-2021, 02:14 PM)snippsat Wrote: company.select_one() was just a demo to get one number to get all most use .select() same as used with companies.
If throw in zip() can do both in one loop.
import requests
from bs4 import BeautifulSoup
import time

url = "https://www.homeadvisor.com/c.Garage-Garage-Doors.Atlanta.GA.-12036.html"
headers = {
    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64)\
     AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.75 Safari/537.36"
}
response = requests.get(url, headers=headers)
soup = BeautifulSoup(response.content, "lxml")
companies = soup.select("div.\@mb-2.md\:\@mb-0.\@hidden.md\:\@block > a > h3")
phone_numbers = soup.select('span.\@ml-1')
for company, phone in zip(companies, phone_numbers):
    print(company.text.strip(), phone.text.strip())
Output:
Doors Over Georgia (678) 798-3712 Tailored Living featuring PremierGarage (404) 946-7940 JJE General Construction (866) 907-7906 Redrock Multi Services, LLC (678) 615-1383 .....
Reply


Messages In This Thread
Scraping Issue with BS - by muzikman - Dec-07-2021, 11:49 AM
RE: Scraping Issue with BS - by snippsat - Dec-07-2021, 04:17 PM
RE: Scraping Issue with BS - by muzikman - Dec-07-2021, 10:36 PM
RE: Scraping Issue with BS - by muzikman - Dec-08-2021, 02:07 PM
RE: Scraping Issue with BS - by snippsat - Dec-08-2021, 02:14 PM
RE: Scraping Issue with BS - by muzikman - Dec-08-2021, 02:19 PM
RE: Scraping Issue with BS - by muzikman - Dec-08-2021, 07:38 PM
RE: Scraping Issue with BS - by muzikman - Dec-08-2021, 09:37 PM
RE: Scraping Issue with BS - by muzikman - Dec-09-2021, 02:09 PM
RE: Scraping Issue with BS - by muzikman - Dec-09-2021, 02:10 PM
RE: Scraping Issue with BS - by snippsat - Dec-09-2021, 04:10 PM
RE: Scraping Issue with BS - by muzikman - Dec-10-2021, 08:49 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Web scraping Possible JavaScript issue johnboy1974 2 2,124 Apr-11-2021, 08:53 AM
Last Post: johnboy1974
  Web scraping: webbrowser issue Truman 10 7,212 Jul-11-2018, 11:57 PM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020