Python Forum
pandas head() not reading all rows
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pandas head() not reading all rows
#1
Hi,

I am reading a csv and applying the def to remove unnecessary data.
If i apply for 174 rows,"dict = (dc_data['Description'].head(174).apply(process_text))" it gives below error.
If i specify 100 rows it works.
Requirements is to apply for all rows.
Any help is appreciated.


Error:
Traceback (most recent call last): File "C:\Python\test\DC\dc_mar2020.py", line 26, in <module> dict = (ec_data['Description'].head(174).apply(process_text)) File "C:\Python\lib\site-packages\pandas\core\series.py", line 3848, in apply mapped = lib.map_infer(values, f, convert=convert_dtype) File "pandas\_libs\lib.pyx", line 2329, in pandas._libs.lib.map_infer File "C:\Python\test\DC\dc_mar2020.py", line 16, in process_text nopunc = [char for char in text if char not in string.punctuation] TypeError: 'float' object is not iterable
Code:-
import pandas as pd
from textblob import TextBlob
import string
import nltk
from nltk.corpus import stopwords

dc_data = pd.read_csv('dc.csv', encoding="ISO-8859-1", index_col=False)
print(dc_data.head())

desc = dc_data['Description']
print(desc.shape)

def process_text(text):
    
    #1
    nopunc = [char for char in text if char not in string.punctuation]
    nopunc = ''.join(nopunc)
    
    #2
    clean_words = [word for word in nopunc.split() if word.lower() not in stopwords.words('english')]
    
    #3
    return clean_words

#Show the Tokenization (a list of tokens )
dict = (dc_data['Description'].head(174).apply(process_text))
print("Dict: ", dict)
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas Dataframe Filtering based on rows mvdlm 0 1,432 Apr-02-2022, 06:39 PM
Last Post: mvdlm
  [Pandas] Help with finding only non-matching rows LowEnd 3 3,551 Nov-12-2021, 02:34 PM
Last Post: jefsummers
  head() d8a988 4 2,649 Sep-03-2020, 09:08 AM
Last Post: buran
  Reading Multiple Sheets using Pandas dhiliptcs 1 4,045 Sep-30-2019, 11:26 PM
Last Post: scidam
  How to add a few empty rows into a pandas dataframe python_newbie09 2 16,331 Sep-20-2019, 08:52 AM
Last Post: python_newbie09
  Subtract rows (like r[1]-r[2] and r[3]-r[3]) and no pandas pradeepkumarbe 1 2,601 Dec-18-2018, 01:16 PM
Last Post: ichabod801
  Write specific rows from pandas dataframe to csv file pradeepkumarbe 3 5,504 Oct-18-2018, 09:33 PM
Last Post: volcano63
  Output substrings from rows in pandas brocq_18 5 3,965 Jun-21-2018, 11:30 AM
Last Post: brocq_18
  Pandas/Excel, reading from one column writing back to another... williamlombard 1 2,700 Mar-01-2018, 06:02 AM
Last Post: williamlombard
  Reading json file as pandas data frame? Alberto 1 8,366 Feb-05-2018, 12:43 AM
Last Post: snippsat

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020