Feb-22-2018, 06:55 PM
Ok - I guess I spoke too soon... Cloud 9 felt really good last night though - thanks Nilamo!
It worked in that it didn't fail in the dreaded red lettering. The problem is I don't have any files. Can you tell me how to save the files to my computer? Also, will this look for all of the completion files there are for a particular well?
Here's what I have so far...
It worked in that it didn't fail in the dreaded red lettering. The problem is I don't have any files. Can you tell me how to save the files to my computer? Also, will this look for all of the completion files there are for a particular well?
Here's what I have so far...
from bs4 import BeautifulSoup import requests import re import urllib import zipfile import os apis = ["49025219260000", "49025059260000", "49025206640000", "49025203350000", "49025213300000", "49025061090000", "49025062840000", ] def wogcc_completions_scraper(): x = 0 while x < len(apis): wogcc_url = """http://wogcc.state.wy.us/wyocompletions.cfm?nApino=""" + str(apis[x][3:10]) print (apis[x]) print str(apis[x][3:10]) las_only = [] wogcc_request = requests.get(wogcc_url) soup = BeautifulSoup(wogcc_request.content, "html.parser") href_tags = soup.find_all('a') ### This section of code will data scrape the WOGCC for the completion report completion_name_regex = "" completion_name_pattern = "" completion_name_regex = re.compile(completion_name_regex) completion_file = re.findall(completion_name_pattern, str(soup)) CNF = completion_name_file = "" CNF1 = [] for q in CNF: q1 = str(q) q2 = q1.decode('unicode_escape').encode('ascii','ignore') q3 = q2.replace(" ","") q4 = q3.replace('"',"") q5 = q4.replace("#","") CNF1.append(q5) b = 0 ### This is a new counter for the CFN1 and CFEL lists. It will keep consistency for when we loop and download logs using this list completion_link_regex = """<a href="http://wugiwus.state.wy.us/(.+?)"><img border="0" height="14" src="search.gif" width="15"/></a>""" completion_link_pattern = re.compile(completion_link_regex) completion_file_end_link = re.findall(completion_link_pattern, str(href_tags)) CFEL = completion_file_end_link RFEL = "" final_completion_link = [] for p in CFEL: final_completion_link.append("http://wugiwus.state.wy.us/" + str(p)) print final_completion_link if final_completion_link == 0: pass else: while b < len(RFEL): ### This loop will loop through everything and download all of the completion reports from the well and name them download1 = requests.get(final_completion_link[b]) with open((str(apis[x]) + "_" + str(CNF1[b].replace("/","")) + ".pdf"), "wb") as code: code.write(download1.content) b += 1 x +=1 wogcc_completions_scraper()