Jun-19-2019, 10:42 PM
Hey I tried making a word counter using a forum page as practice to just isolate all the thread titles of a given page and then separate each word onto a new line. However, I cannot figure out how to get rid of extra info given in HTML, as I just want the thread titles. I thought using span.string instead of str(span) would solve this but it just gives an error.
import requests from bs4 import BeautifulSoup import operator def start(url): word_list = [] source_code = requests.get(url).text soup = BeautifulSoup(source_code, "html.parser") for span in soup.find_all('tr', {'class': 'inline_row'}): content = str(span) words = content.lower().split() for each_word in words: print(each_word) word_list.append(each_word) start('https://python-forum.io/Forum-Data-Science')output:
"C:\Users\Jake\PycharmProjects\practice baby\venv\Scripts\python.exe" "C:/Users/Jake/PycharmProjects/practice baby/classes adn objects.py" <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hello. i'm so sorry in advance. i am super new to this. if this question isn't allowed, please discard. i have something as follows: sample001a.com, -12, sample002b.com, -16, sample002c.com, -18 sa..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-aligning-data?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <img alt="" border="0" src="images/solved.png" style="vertical-align: middle;"/> <span class="subject_new" id="tid_19193"><a href="thread-aligning-data">aligning data</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-gat">gat</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19193);">7</a></td> <td align="center" class="trow1 forumdisplay_regular">140</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19193"> <ul class="star_rating star_rating_notrated" id="rating_thread_19193"> <li class="current_rating" id="current_rating_19193" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19193, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="(37 minutes ago)vindo wrote: 2) is your second set the expectation of the outcome ? i mistyped. my expected outcome is to align it by sample number. i've been able to extract the sample number using..."> <span class="lastpost smalltext">7 hours ago<br/> <a href="thread-aligning-data?action=lastpost">last post</a>: <a href="https://python-forum.io/user-gat">gat</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hi, i have below pandas dataframe: trial1 trial1 trial1 trial2 trial2 name sub_item1 sub_item2 sub_item3 sub_item4 sub_item5 2019-06-01 2016-06-01 2019-06-01 2019-06-01 2019-06-01 vba 1 0 0 1 1 ..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-pandas-how-to-re-arrange-dataframe-columns?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19216"><a href="thread-pandas-how-to-re-arrange-dataframe-columns">[pandas] how to re-arrange dataframe columns</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-srimekala">srimekala</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19216);">3</a></td> <td align="center" class="trow2 forumdisplay_regular">113</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19216"> <ul class="star_rating star_rating_notrated" id="rating_thread_19216"> <li class="current_rating" id="current_rating_19216" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19216, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="it gives some error: i use python3.6: typeerror: '>' not supported between instances of 'str' and 'int'"> <span class="lastpost smalltext">10 hours ago<br/> <a href="thread-pandas-how-to-re-arrange-dataframe-columns?action=lastpost">last post</a>: <a href="https://python-forum.io/user-srimekala">srimekala</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hello. i'm so sorry in advance. i am super new to this. if this question isn't allowed, please discard. i have something as follows: sample001a.com, -12 sample002a.com, -14 sample003a.com"> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-aligning-excel-data?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <img alt="" border="0" src="images/solved.png" style="vertical-align: middle;"/> <span class="subject_new" id="tid_19192"><a href="thread-aligning-excel-data">aligning excel data</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-gat">gat</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19192);">1</a></td> <td align="center" class="trow1 forumdisplay_regular">87</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19192"> <ul class="star_rating star_rating_notrated" id="rating_thread_19192"> <li class="current_rating" id="current_rating_19192" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19192, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="did you accidentally post only part of the message?"> <span class="lastpost smalltext">jun-17-2019, 07:05 pm<br/> <a href="thread-aligning-excel-data?action=lastpost">last post</a>: <a href="https://python-forum.io/user-michalmonday">michalmonday</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="situation: learning pandas, using nederlands ov chipkaart data (mine) for a month to test pandas and practice currently can read the file into a dataframe. quote:date, check-in, departure, check-o..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-two-dataframes-merged?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <img alt="" border="0" src="images/solved.png" style="vertical-align: middle;"/> <span class="subject_new" id="tid_18871"><a href="thread-two-dataframes-merged">two dataframes merged</a></span><!-- start: forumdisplay_thread_multipage --> <span class="smalltext">(pages: <!-- start: forumdisplay_thread_multipage_page --> <a href="thread-two-dataframes-merged">1</a> <!-- end: forumdisplay_thread_multipage_page --><!-- start: forumdisplay_thread_multipage_page --> <a href="thread-two-dataframes-merged?page=2">2</a> <!-- end: forumdisplay_thread_multipage_page -->)</span> <!-- end: forumdisplay_thread_multipage --></span> <div class="author smalltext"><a href="https://python-forum.io/user-ecniv">ecniv</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(18871);">10</a></td> <td align="center" class="trow2 forumdisplay_regular">313</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_18871"> <ul class="star_rating star_rating_notrated" id="rating_thread_18871"> <li class="current_rating" id="current_rating_18871" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(18871, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="quote: date check-in departure check-out \ 0 04-05-2019 09:53 ouderkerk a/d amstel, ouderkerk a/d amst nan 1 04-05-2019 nan ouderkerk a/d amstel, ouderkerk a/d amst 10:02 2 04-05-2019 13:16 amste..."> <span class="lastpost smalltext">jun-16-2019, 09:10 pm<br/> <a href="thread-two-dataframes-merged?action=lastpost">last post</a>: <a href="https://python-forum.io/user-ecniv">ecniv</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="i am trying to solve a problem, but i am new to really complicated coding. below is the problem summary and code i tried. i will be very thankful if someone can provide some advice on this. i have 2..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-distance-between-2-user-defined-geo-grids-in-km?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19174"><a href="thread-distance-between-2-user-defined-geo-grids-in-km">distance between 2 user defined geo-grids in km</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-chandrakant98c">chandrakant98c</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19174);">0</a></td> <td align="center" class="trow1 forumdisplay_regular">77</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19174"> <ul class="star_rating star_rating_notrated" id="rating_thread_19174"> <li class="current_rating" id="current_rating_19174" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19174, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="i am trying to solve a problem, but i am new to really complicated coding. below is the problem summary and code i tried. i will be very thankful if someone can provide some advice on this. i have 2..."> <span class="lastpost smalltext">jun-16-2019, 11:26 am<br/> <a href="thread-distance-between-2-user-defined-geo-grids-in-km?action=lastpost">last post</a>: <a href="https://python-forum.io/user-chandrakant98c">chandrakant98c</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hello, i have a data frame df_train which has a column sub_division. the values in the column is look like below abc_commercial abc_private test rom div rom div test sec rom i am trying to 1..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-pandas-convert-categorical-data-to-numbers?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19151"><a href="thread-pandas-convert-categorical-data-to-numbers">[pandas] convert categorical data to numbers</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-pradeep-as400">pradeep_as400</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19151);">1</a></td> <td align="center" class="trow2 forumdisplay_regular">164</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19151"> <ul class="star_rating star_rating_notrated" id="rating_thread_19151"> <li class="current_rating" id="current_rating_19151" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19151, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="a possibility that might be useful for you: import pandas as pd s = pd.series(['abc_commercial', 'abc_private', 'test rom div', 'rom div', 'test sec rom'], dtype="object") df = pd.dataframe(s, col..."> <span class="lastpost smalltext">jun-15-2019, 08:27 am<br/> <a href="thread-pandas-convert-categorical-data-to-numbers?action=lastpost">last post</a>: <a href="https://python-forum.io/user-thomasl">thomasl</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hello. i am trying to do a market basket analysis of transaction data. so far i have been able to organize the transaction data into a pandas dataframe: #import libraries import pandas as pd #load..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-market-basket-analysis-finding-association-rules?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19142"><a href="thread-market-basket-analysis-finding-association-rules">market basket analysis: finding association rules</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-kylenater">kylenater</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19142);">0</a></td> <td align="center" class="trow1 forumdisplay_regular">92</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19142"> <ul class="star_rating star_rating_notrated" id="rating_thread_19142"> <li class="current_rating" id="current_rating_19142" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19142, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="hello. i am trying to do a market basket analysis of transaction data. so far i have been able to organize the transaction data into a pandas dataframe: #import libraries import pandas as pd #load..."> <span class="lastpost smalltext">jun-14-2019, 07:10 pm<br/> <a href="thread-market-basket-analysis-finding-association-rules?action=lastpost">last post</a>: <a href="https://python-forum.io/user-kylenater">kylenater</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hi all i have a dataset where the -1 means the point where i need to stop reading. for example the dataset looks like that: 0 1 2 3 4 5 0 58 68 58 59 -1 -1 1 59 69 59 -1 -1 -1 2 93 94 93 33 -..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-pandas-find-the-first-element-that-is-1?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19077"><a href="thread-pandas-find-the-first-element-that-is-1">[pandas] find the first element that is -1</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-dervast">dervast</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19077);">3</a></td> <td align="center" class="trow2 forumdisplay_regular">215</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19077"> <ul class="star_rating star_rating_notrated" id="rating_thread_19077"> <li class="current_rating" id="current_rating_19077" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19077, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="you can do this very fast with numpy import numpy as np a = np.array([[58, 68, 58, 59, -1, -1], [59, 69, 59, -1, -1, -1], [93, 94, 93, 33, -1, -1], [58, 59, 58, 68, -1, -1], [92, 94, 92, 33, -1..."> <span class="lastpost smalltext">jun-14-2019, 04:58 pm<br/> <a href="thread-pandas-find-the-first-element-that-is-1?action=lastpost">last post</a>: <a href="https://python-forum.io/user-thomasl">thomasl</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hi everyone, (disclaimer: i am quite a beginner onpython and data science in general… **smile** ) i have designed and published my 1st kernel on kaggle (here: https://www.kaggle.com/plbescond/quali..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-optimizing-a-model-output-y-with-some-x-parameters-defined?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19130"><a href="thread-optimizing-a-model-output-y-with-some-x-parameters-defined">optimizing a model output (y) with some (x) parameters defined</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-pierrelouisbescond">pierrelouisbescond</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19130);">0</a></td> <td align="center" class="trow1 forumdisplay_regular">63</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19130"> <ul class="star_rating star_rating_notrated" id="rating_thread_19130"> <li class="current_rating" id="current_rating_19130" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19130, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="hi everyone, (disclaimer: i am quite a beginner onpython and data science in general… **smile** ) i have designed and published my 1st kernel on kaggle (here: https://www.kaggle.com/plbescond/quali..."> <span class="lastpost smalltext">jun-14-2019, 10:28 am<br/> <a href="thread-optimizing-a-model-output-y-with-some-x-parameters-defined?action=lastpost">last post</a>: <a href="https://python-forum.io/user-pierrelouisbescond">pierrelouisbescond</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hello! i'm new here and i'm a computer science student focusing on natural language processing. a few months ago i got a project from my campus to make a project related to nlp. the project should be..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-suggestion-needed-for-natural-language-processing-project?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19109"><a href="thread-suggestion-needed-for-natural-language-processing-project">suggestion needed for natural language processing project</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-jojo1268">jojo1268</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19109);">3</a></td> <td align="center" class="trow2 forumdisplay_regular">124</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19109"> <ul class="star_rating star_rating_notrated" id="rating_thread_19109"> <li class="current_rating" id="current_rating_19109" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19109, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="(11 hours ago)joe_momma wrote: sounds like you have not read the nltk book. in idle: import nltk help(nltk) check out the description. the book tells you how to get some data sets and what the librar..."> <span class="lastpost smalltext">jun-14-2019, 09:24 am<br/> <a href="thread-suggestion-needed-for-natural-language-processing-project?action=lastpost">last post</a>: <a href="https://python-forum.io/user-jojo1268">jojo1268</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hi i have a data frame with strings that look like date and another column of time. what would be the best way to convert them to actual datetime objects (or to two objects of date and time) ? not..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-simple-string-to-time-within-a-pandas-dataframe?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <img alt="" border="0" src="images/solved.png" style="vertical-align: middle;"/> <span class="subject_new" id="tid_19119"><a href="thread-simple-string-to-time-within-a-pandas-dataframe">simple string to time within a pandas dataframe</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-ecniv">ecniv</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19119);">1</a></td> <td align="center" class="trow1 forumdisplay_regular">127</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19119"> <ul class="star_rating star_rating_notrated" id="rating_thread_19119"> <li class="current_rating" id="current_rating_19119" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19119, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="you can use pd.to_datetime utility function, e.g. import pandas as pd df = pd.dataframe({'atime': ['13-06-2019 10:00', '12-06-2019 09:15'], 'x': [1, 2]}) df.atime = pd.to_datetime(df.atime)"> <span class="lastpost smalltext">jun-14-2019, 03:25 am<br/> <a href="thread-simple-string-to-time-within-a-pandas-dataframe?action=lastpost">last post</a>: <a href="https://python-forum.io/user-scidam">scidam</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="i am trying to color sections of this graph by dividing it into times from dawn to sunset and from sunset to dawn (next day). the graph looks (almost) exactly the way i want,but when i uncomment lin..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-span-color-on-graph-sunset-dawn?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19100"><a href="thread-span-color-on-graph-sunset-dawn">span color on graph sunset/dawn</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-mcgrim">mcgrim</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19100);">0</a></td> <td align="center" class="trow2 forumdisplay_regular">101</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19100"> <ul class="star_rating star_rating_notrated" id="rating_thread_19100"> <li class="current_rating" id="current_rating_19100" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19100, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="i am trying to color sections of this graph by dividing it into times from dawn to sunset and from sunset to dawn (next day). the graph looks (almost) exactly the way i want,but when i uncomment lin..."> <span class="lastpost smalltext">jun-13-2019, 09:22 am<br/> <a href="thread-span-color-on-graph-sunset-dawn?action=lastpost">last post</a>: <a href="https://python-forum.io/user-mcgrim">mcgrim</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="the list comprehension in this code (see last line) import datetime from datetime import timezone from datetime import timedelta from datetime import time import matplotlib.dates as dates custo..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-list-comprehension-invalid-syntax?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19082"><a href="thread-list-comprehension-invalid-syntax">list comprehension invalid syntax</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-mcgrim">mcgrim</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(19082);">1</a></td> <td align="center" class="trow1 forumdisplay_regular">87</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_19082"> <ul class="star_rating star_rating_notrated" id="rating_thread_19082"> <li class="current_rating" id="current_rating_19082" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19082, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="a=[x[i].time() for i in range(10)]"> <span class="lastpost smalltext">jun-12-2019, 08:28 pm<br/> <a href="thread-list-comprehension-invalid-syntax?action=lastpost">last post</a>: <a href="https://python-forum.io/user-yoriz">yoriz</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hello, i need to create an algorithm that can generate all possible protein interaction chains of size x (x proteins in the chain). i have a test file that represents binary interactions (1 interac..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-protein-interaction-chains?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19072"><a href="thread-protein-interaction-chains">protein interaction chains</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-amniote">amniote</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19072);">2</a></td> <td align="center" class="trow2 forumdisplay_regular">98</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19072"> <ul class="star_rating star_rating_notrated" id="rating_thread_19072"> <li class="current_rating" id="current_rating_19072" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19072, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="(49 minutes ago)perfringo wrote: general and probably stupid question: is there 1 -> 4 relationship or 1 <-> 4 in the example file, we find 1 4 which corresponds to 1 -> 4. we find lower 4 1 which c..."> <span class="lastpost smalltext">jun-12-2019, 03:32 pm<br/> <a href="thread-protein-interaction-chains?action=lastpost">last post</a>: <a href="https://python-forum.io/user-amniote">amniote</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="i have two pandas data frames, one empirical-derived daily data (rows are dates, columns are meteo stations), the other df is model output of same info. i need to statistically compare the two to see..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-statistical-analysis-of-two-dataframes?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_13389"><a href="thread-statistical-analysis-of-two-dataframes">statistical analysis of two dataframes</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-zhl">zhl</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(13389);">1</a></td> <td align="center" class="trow1 forumdisplay_regular">340</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_13389"> <ul class="star_rating star_rating_notrated" id="rating_thread_13389"> <li class="current_rating" id="current_rating_13389" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(13389, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="wouldnt you do just make a new df with an if statement comparison... compare dataframes equals or different"> <span class="lastpost smalltext">jun-11-2019, 07:26 pm<br/> <a href="thread-statistical-analysis-of-two-dataframes?action=lastpost">last post</a>: <a href="https://python-forum.io/user-ecniv">ecniv</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="i have the below data that l would like to process, can anyone help with the python code for it? i just want to increment every column with 100. preferably with the openpyxl or pandas packages. fo..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-use-python-packages-to-increment-a-column-not-a-row?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_19012"><a href="thread-use-python-packages-to-increment-a-column-not-a-row">use python packages to increment a column(not a row)</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-ricmacharia">ricmacharia</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(19012);">0</a></td> <td align="center" class="trow2 forumdisplay_regular">122</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_19012"> <ul class="star_rating star_rating_notrated" id="rating_thread_19012"> <li class="current_rating" id="current_rating_19012" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(19012, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="i have the below data that l would like to process, can anyone help with the python code for it? i just want to increment every column with 100. preferably with the openpyxl or pandas packages. fo..."> <span class="lastpost smalltext">jun-10-2019, 09:57 am<br/> <a href="thread-use-python-packages-to-increment-a-column-not-a-row?action=lastpost">last post</a>: <a href="https://python-forum.io/user-ricmacharia">ricmacharia</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="we (a fellow architecture student and i) want to develop a tool which can autonomously layout kitchen interiors based on user input parameters such as cooking habits, room dimensions etc. we will ha..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-scikit-vs-dynamo-vs-grasshopper-dodo-for-a-i-planner-project?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_18972"><a href="thread-scikit-vs-dynamo-vs-grasshopper-dodo-for-a-i-planner-project">scikit vs dynamo vs grasshopper/dodo for a.i. planner project</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-julian">julian</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(18972);">1</a></td> <td align="center" class="trow1 forumdisplay_regular">154</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_18972"> <ul class="star_rating star_rating_notrated" id="rating_thread_18972"> <li class="current_rating" id="current_rating_18972" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(18972, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="there are over 10,000 python packages available for machine learning. perhaps you should browse through and see if there's anything that can help with your project, see: https://pypi.org/search/?q=m..."> <span class="lastpost smalltext">jun-10-2019, 02:41 am<br/> <a href="thread-scikit-vs-dynamo-vs-grasshopper-dodo-for-a-i-planner-project?action=lastpost">last post</a>: <a href="https://python-forum.io/user-larz60">larz60+</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="i am trying to put in a bar graph some values stored in a file and the times when these values took place. the bar chart looks like the way i want as long as i have it without dates, but once i intr..."> <!-- start: forumdisplay_thread_attachment_count --> <div class="float_right"><img alt="" src="https://python-forum.io/images/paperclip.png" title="this thread contains 1 attachment."/></div> <!-- end: forumdisplay_thread_attachment_count --> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-xticks?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_18931"><a href="thread-xticks">xticks</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-mcgrim">mcgrim</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(18931);">5</a></td> <td align="center" class="trow2 forumdisplay_regular">411</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_18931"> <ul class="star_rating star_rating_notrated" id="rating_thread_18931"> <li class="current_rating" id="current_rating_18931" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(18931, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="i am now working with the same graph, but i would like to get the hours (every 3) between those dates, but once i write this lines time_list=[ceu_times[position1].hour() + datetime.timedelta(hours=..."> <span class="lastpost smalltext">jun-09-2019, 09:00 pm<br/> <a href="thread-xticks?action=lastpost">last post</a>: <a href="https://python-forum.io/user-mcgrim">mcgrim</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow1 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow1 forumdisplay_regular" width="2%"> </td> <td class="trow1 forumdisplay_regular" title="hi all, i have a problem when using the code for webscraping. appreciate any help please. i get an error: error: --------------------------------------------------------------------------- typee..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-google-webscraper?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_18859"><a href="thread-google-webscraper">google webscraper</a></span><!-- start: forumdisplay_thread_multipage --> <span class="smalltext">(pages: <!-- start: forumdisplay_thread_multipage_page --> <a href="thread-google-webscraper">1</a> <!-- end: forumdisplay_thread_multipage_page --><!-- start: forumdisplay_thread_multipage_page --> <a href="thread-google-webscraper?page=2">2</a> <!-- end: forumdisplay_thread_multipage_page -->)</span> <!-- end: forumdisplay_thread_multipage --></span> <div class="author smalltext"><a href="https://python-forum.io/user-kirito85">kirito85</a></div> </div> </td> <td align="center" class="trow1 forumdisplay_regular"><a href="javascript:mybb.whoposted(18859);">12</a></td> <td align="center" class="trow1 forumdisplay_regular">303</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow1 forumdisplay_regular" id="rating_table_18859"> <ul class="star_rating star_rating_notrated" id="rating_thread_18859"> <li class="current_rating" id="current_rating_18859" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(18859, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow1 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="you are welcome."> <span class="lastpost smalltext">jun-07-2019, 08:24 am<br/> <a href="thread-google-webscraper?action=lastpost">last post</a>: <a href="https://python-forum.io/user-heiner55">heiner55</a></span> </td> </tr> <tr class="inline_row"> <td align="center" class="trow2 forumdisplay_regular" width="2%"><span class="thread_status newfolder" title="new posts."> </span></td> <td align="center" class="trow2 forumdisplay_regular" width="2%"> </td> <td class="trow2 forumdisplay_regular" title="hello! i have a timeserie of values where every week is devided into 4 parts, each representing 42 hours. is it possible to convert this to hourly values using pandas built in functions? the value..."> <div> <span> <!-- start: forumdisplay_thread_gotounread --> <a href="thread-custom-timeinterval-converted-to-hourly-values-using-pandas?action=newpost"><img alt="go to first unread post" src="https://python-forum.io/images/jump.png" title="go to first unread post"/></a> <!-- end: forumdisplay_thread_gotounread --> <span class="subject_new" id="tid_18930"><a href="thread-custom-timeinterval-converted-to-hourly-values-using-pandas">custom timeinterval converted to hourly values using pandas?</a></span></span> <div class="author smalltext"><a href="https://python-forum.io/user-sinpy">sinpy</a></div> </div> </td> <td align="center" class="trow2 forumdisplay_regular"><a href="javascript:mybb.whoposted(18930);">1</a></td> <td align="center" class="trow2 forumdisplay_regular">240</td> <!-- start: forumdisplay_thread_rating --> <td align="center" class="trow2 forumdisplay_regular" id="rating_table_18930"> <ul class="star_rating star_rating_notrated" id="rating_thread_18930"> <li class="current_rating" id="current_rating_18930" style="width: 0%">0 vote(s) - 0 out of 5 in average</li> </ul> <script type="text/javascript"> <!-- rating.build_forumdisplay(18930, { width: '0', extra_class: ' star_rating_notrated', current_average: '0 vote(s) - 0 out of 5 in average' }); // --> </script> </td> <!-- end: forumdisplay_thread_rating --> <td class="trow2 forumdisplay_regular" style="white-space: nowrap; text-align: right;" title="if you are totally new to python, you should begin with a tutorial: https://pandas.pydata.org/pandas-docs/st...10min.html"> <span class="lastpost smalltext">jun-07-2019, 05:06 am<br/> <a href="thread-custom-timeinterval-converted-to-hourly-values-using-pandas?action=lastpost">last post</a>: <a href="https://python-forum.io/user-heiner55">heiner55</a></span> </td> </tr> Process finished with exit code 0Thanks for any help