Python Forum

Full Version: [BS4] Having Trouble Scraping A Single URL
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
It seems this should work, not sure why I'm having issues Wall LOL

I'm trying to scrape a single URL off a page.. it's an IFRAME SRC link.

from bs4 import BeautifulSoup
import urllib.request
import re

scrape = urllib.request.urlopen('https://pingler.com/').read()
soup = BeautifulSoup(scrape, 'html.parser')

#Things I've tried.....

for zzz in soup.find_all(re.compile("^iframe")):
print(zzz.src)

soup.find_all('iframe', src=re.compile('https://api-secure.solvemedia.com'))
print(soup)

soup.select('iframe[src^="https://api-secure.solvemedia.com"]')
print(soup)
Any ideas where I'm going wrong?

I want to scrape the full URL without any tags.

Example:

https://api-secure.solvemedia.com/papi/challenge.noscript?k=yBDQMU1DZT1vWxD.NwzBVCVe3-2lB6kO


UPDATE: Kept messing with it and got something to work.

for elem in soup.find_all('iframe', src=re.compile('https://api-secure\.solvemedia\.com')):
    print (elem['src'])