Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 pandas head() not reading all rows
#1
Hi,

I am reading a csv and applying the def to remove unnecessary data.
If i apply for 174 rows,"dict = (dc_data['Description'].head(174).apply(process_text))" it gives below error.
If i specify 100 rows it works.
Requirements is to apply for all rows.
Any help is appreciated.


Error:
Traceback (most recent call last): File "C:\Python\test\DC\dc_mar2020.py", line 26, in <module> dict = (ec_data['Description'].head(174).apply(process_text)) File "C:\Python\lib\site-packages\pandas\core\series.py", line 3848, in apply mapped = lib.map_infer(values, f, convert=convert_dtype) File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer File "C:\Python\test\DC\dc_mar2020.py", line 16, in process_text nopunc = [char for char in text if char not in string.punctuation] TypeError: 'float' object is not iterable
Code:-
import pandas as pd
from textblob import TextBlob
import string
import nltk
from nltk.corpus import stopwords

dc_data = pd.read_csv('dc.csv', encoding="ISO-8859-1", index_col=False)
print(dc_data.head())

desc = dc_data['Description']
print(desc.shape)

def process_text(text):
    
    #1
    nopunc = [char for char in text if char not in string.punctuation]
    nopunc = ''.join(nopunc)
    
    #2
    clean_words = [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]
    
    #3
    return clean_words

#Show the Tokenization (a list of tokens )
dict = (dc_data['Description'].head(174).apply(process_text))
print("Dict: ", dict)
Larz60+ wrote Apr-07-2020, 07:52 PM:
Please post all code, output and errors (in it's entirety) between their respective tags. I did it for you this time, Here are instructions on how to do it yourself next time.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Reading Multiple Sheets using Pandas dhiliptcs 1 848 Sep-30-2019, 11:26 PM
Last Post: scidam
  How to add a few empty rows into a pandas dataframe python_newbie09 2 3,658 Sep-20-2019, 08:52 AM
Last Post: python_newbie09
  Subtract rows (like r[1]-r[2] and r[3]-r[3]) and no pandas pradeepkumarbe 1 655 Dec-18-2018, 01:16 PM
Last Post: ichabod801
  Write specific rows from pandas dataframe to csv file pradeepkumarbe 3 1,347 Oct-18-2018, 09:33 PM
Last Post: volcano63
  Output substrings from rows in pandas brocq_18 5 948 Jun-21-2018, 11:30 AM
Last Post: brocq_18
  Pandas/Excel, reading from one column writing back to another... williamlombard 1 891 Mar-01-2018, 06:02 AM
Last Post: williamlombard
  Reading json file as pandas data frame? Alberto 1 5,902 Feb-05-2018, 12:43 AM
Last Post: snippsat
  pandas restricting csv read to certain rows metalray 5 15,317 Dec-16-2017, 07:39 AM
Last Post: metalray

Forum Jump:


Users browsing this thread: 1 Guest(s)