Python Forum

Full Version: Python Web Scrapping
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I am in the process of creating a web scrapping tool from scratch that I will use at my job. I work at a medical transportation company and will be scraping a website for trip information and saving it to a database. I am new to programming and felt for a "real project" it would be something interesting to tackle. I have been studying python for about 3 weeks. 2 hours a day to give you a background on my knowledge.



session.post('https://www.medanswering.com/login.taf?_function=check', params=payload)
r = session.get('https://www.medanswering.com/admintrips.taf?_function=detail&Trip_Auth_ID=' + invoice)
html_content = r.content
html_content2 = r.text
soup = BeautifulSoup(html_content, 'html.parser')
soup2 = BeautifulSoup(html_content2, 'html.parser')


#Grabbing the Date and time
PU_time_finder = soup2.find_all("td")[100]
print(PU_time_finder.text)


Whats happening is I obtain the data I need. However, I am retrieving it from a table on a website and the date and time I need are in the same "td" tag. Causing the output below

10/24/17
9:30 am



It appears in my console exactly like that. I thought "hmm maybe I can put it in a list" So i tried and it didnt work. I am sure there is more code needed to manipulate it a little more. I am using the libraries "requests" and "BeautifulSoup4".

P.S I did not incorporate the authentication part of my code showing the Username and password. That part is functioning correctly.
please, rather than posting your code in bold, use code tags see BBCODE
when pasting code, use shift-ctrl-v to preserve indentation
Quote:and the date and time I need are in the same "td" tag.
Whats the exact html of the td code for that section?
I actually thought of a solution and was able to implement it. What I did was convert it to a list and removed the characters that way and converted it back to a string. Very excited it worked!