InvalidSchema(“No connection adapters were found for {!r}”.format(url)) - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: Web Scraping & Web Development (https://python-forum.io/forum-13.html) +--- Thread: InvalidSchema(“No connection adapters were found for {!r}”.format(url)) (/thread-33939.html) |
InvalidSchema(“No connection adapters were found for {!r}”.format(url)) - rhat398 - Jun-11-2021 What does this error means and how to address this kind of error?I am getting this error when I try to parse the links for title.I am trying to scrape with requests post method here is the code:import requests from bs4 import BeautifulSoup headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'Accept-Language': 'en-US,en;q=0.5', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'X-Requested-With': 'XMLHttpRequest', 'Origin': 'https://lifebridgecapital.com', 'Connection': 'keep-alive', 'Referer': 'https://lifebridgecapital.com/podcast/', 'Sec-GPC': '1', 'TE': 'Trailers', } data = {'action': 'gdlr_core_post_ajax', 'settings[category][]': 'podcast', 'settings[tag]': '', 'settings[num-fetch]': '9', 'settings[paged]': '1', 'option[name]': 'paged', } session = requests.Session() for page in range(0, 55): data['option[value]'] = str(page + 1) response = session.post('https://lifebridgecapital.com/wp-admin/admin-ajax.php', headers=headers, data=data) links = [a['href'] for a in BeautifulSoup(response.text, 'lxml').select('h3 > a')] for link in links: response = session.get(link) page = BeautifulSoup(response.text, 'lxml') title = page.find('h3').text print(f'Title: {title}, Link: {link}') #print(f'title: {title}, links: {links}')I am getting all the links but when try to parse that link for title this Invalid Schema error occurs I searched alot on google before asking here at SO but didn't get the solution or answer to why this error occurring . |