Jun-11-2021, 08:40 PM
What does this error means and how to address this kind of error?I am getting this error
Error:Traceback (most recent call last):
File "load-more.py", line 146, in <module>
response = session.get(link)
File "C:\Users\Xone\.virtualenvs\Web_Scrapers-A6P4QRzc\lib\site-packages\requests \sessions.py", line 555, in get
return self.request('GET', url, **kwargs)
File "C:\Users\Xone\.virtualenvs\Web_Scrapers-A6P4QRzc\lib\site-packages\requests\sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "C:\Users\Xone\.virtualenvs\Web_Scrapers-A6P4QRzc\lib\site-packages\requests\sessions.py", line 649, in send
adapter = self.get_adapter(url=request.url)
File "C:\Users\Xone\.virtualenvs\Web_Scrapers-A6P4QRzc\lib\site-packages\requests\sessions.py", line 742, in get_adapter
raise InvalidSchema("No connection adapters were found for {!r}".format(url))
requests.exceptions.InvalidSchema: No connection adapters were found for '\\"https:\\/\\/lifebridgecapital.com\\/2021\\/06\\/11\\/ws964-multifamily-investing-is-a-team-sport-with-cameron-roy\\/\\"'
when I try to parse the links for title.I am trying to scrape with requests post method here is the code:import requests from bs4 import BeautifulSoup headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:89.0) Gecko/20100101 Firefox/89.0', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'Accept-Language': 'en-US,en;q=0.5', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'X-Requested-With': 'XMLHttpRequest', 'Origin': 'https://lifebridgecapital.com', 'Connection': 'keep-alive', 'Referer': 'https://lifebridgecapital.com/podcast/', 'Sec-GPC': '1', 'TE': 'Trailers', } data = {'action': 'gdlr_core_post_ajax', 'settings[category][]': 'podcast', 'settings[tag]': '', 'settings[num-fetch]': '9', 'settings[paged]': '1', 'option[name]': 'paged', } session = requests.Session() for page in range(0, 55): data['option[value]'] = str(page + 1) response = session.post('https://lifebridgecapital.com/wp-admin/admin-ajax.php', headers=headers, data=data) links = [a['href'] for a in BeautifulSoup(response.text, 'lxml').select('h3 > a')] for link in links: response = session.get(link) page = BeautifulSoup(response.text, 'lxml') title = page.find('h3').text print(f'Title: {title}, Link: {link}') #print(f'title: {title}, links: {links}')I am getting all the links but when try to parse that link for title this Invalid Schema error occurs I searched alot on google before asking here at SO but didn't get the solution or answer to why this error occurring .