Python Forum
pandas head() not reading all rows
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pandas head() not reading all rows
#1
Hi,

I am reading a csv and applying the def to remove unnecessary data.
If i apply for 174 rows,"dict = (dc_data['Description'].head(174).apply(process_text))" it gives below error.
If i specify 100 rows it works.
Requirements is to apply for all rows.
Any help is appreciated.


Error:
Traceback (most recent call last): File "C:\Python\test\DC\dc_mar2020.py", line 26, in <module> dict = (ec_data['Description'].head(174).apply(process_text)) File "C:\Python\lib\site-packages\pandas\core\series.py", line 3848, in apply mapped = lib.map_infer(values, f, convert=convert_dtype) File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer File "C:\Python\test\DC\dc_mar2020.py", line 16, in process_text nopunc = [char for char in text if char not in string.punctuation] TypeError: 'float' object is not iterable
Code:-
import pandas as pd
from textblob import TextBlob
import string
import nltk
from nltk.corpus import stopwords

dc_data = pd.read_csv('dc.csv', encoding="ISO-8859-1", index_col=False)
print(dc_data.head())

desc = dc_data['Description']
print(desc.shape)

def process_text(text):
    
    #1
    nopunc = [char for char in text if char not in string.punctuation]
    nopunc = ''.join(nopunc)
    
    #2
    clean_words = [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]
    
    #3
    return clean_words

#Show the Tokenization (a list of tokens )
dict = (dc_data['Description'].head(174).apply(process_text))
print("Dict: ", dict)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  head() d8a988 4 520 Sep-03-2020, 09:08 AM
Last Post: buran
  Reading Multiple Sheets using Pandas dhiliptcs 1 1,254 Sep-30-2019, 11:26 PM
Last Post: scidam
  How to add a few empty rows into a pandas dataframe python_newbie09 2 5,908 Sep-20-2019, 08:52 AM
Last Post: python_newbie09
  Subtract rows (like r[1]-r[2] and r[3]-r[3]) and no pandas pradeepkumarbe 1 811 Dec-18-2018, 01:16 PM
Last Post: ichabod801
  Write specific rows from pandas dataframe to csv file pradeepkumarbe 3 1,785 Oct-18-2018, 09:33 PM
Last Post: volcano63
  Output substrings from rows in pandas brocq_18 5 1,107 Jun-21-2018, 11:30 AM
Last Post: brocq_18
  Pandas/Excel, reading from one column writing back to another... williamlombard 1 1,092 Mar-01-2018, 06:02 AM
Last Post: williamlombard
  Reading json file as pandas data frame? Alberto 1 6,333 Feb-05-2018, 12:43 AM
Last Post: snippsat
  pandas restricting csv read to certain rows metalray 5 16,059 Dec-16-2017, 07:39 AM
Last Post: metalray

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020