Python Forum
[BS4] Having Trouble Scraping A Single URL
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[BS4] Having Trouble Scraping A Single URL
#1
It seems this should work, not sure why I'm having issues Wall LOL

I'm trying to scrape a single URL off a page.. it's an IFRAME SRC link.

from bs4 import BeautifulSoup
import urllib.request
import re

scrape = urllib.request.urlopen('https://pingler.com/').read()
soup = BeautifulSoup(scrape, 'html.parser')

#Things I've tried.....

for zzz in soup.find_all(re.compile("^iframe")):
print(zzz.src)

soup.find_all('iframe', src=re.compile('https://api-secure.solvemedia.com'))
print(soup)

soup.select('iframe[src^="https://api-secure.solvemedia.com"]')
print(soup)
Any ideas where I'm going wrong?

I want to scrape the full URL without any tags.

Example:

https://api-secure.solvemedia.com/papi/challenge.noscript?k=yBDQMU1DZT1vWxD.NwzBVCVe3-2lB6kO


UPDATE: Kept messing with it and got something to work.

for elem in soup.find_all('iframe', src=re.compile('https://api-secure\.solvemedia\.com')):
    print (elem['src'])
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
Thumbs Up Issue facing while scraping the data from different websites in single script. Balamani 1 2,108 Oct-20-2020, 09:56 AM
Last Post: Larz60+
  Scraping from multiple URLS to print in a single line. jb89 4 3,354 Jan-29-2020, 06:12 AM
Last Post: perfringo

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020