Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Using chunk size issue
#1
Hi ,

My inputs

query_string= sel * from table ( 340k rows)
I am using
df = pd.read_sql(query_string ,conn_bmg,chunksize=50000)
and working on

using this code
total=0
for each in df:
    df1= each[each.Column.str.contains("VENMO")]
    df1['column']='VENMO'


df1.to_csv(r"C:\Users\xxxx\Desktop\venmo_output8.CSV", index=False)
I need to get 340k rows.. since all 340k rows have VENMO. BUt I am getting only 27k rows.
I don't know why. Do I am using any thing wrong in code?
Reply
#2
Any help on this?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Why doesnt chunk generation work? LotosProgramer 1 1,908 Apr-02-2022, 08:25 AM
Last Post: deanhystad
  size of set vs size of dict zweb 0 2,116 Oct-11-2019, 01:32 AM
Last Post: zweb
  lambda layer size issue lambdabeginer 1 1,923 Aug-26-2019, 07:23 PM
Last Post: Larz60+
  CSV file created is huge in size. How to reduce the size? pramoddsrb 0 10,432 Apr-26-2018, 12:38 AM
Last Post: pramoddsrb

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020