My python code is running very slow on millions of records

Thread Rating:

0 Vote(s) - 0 Average
1
2
3
4
5

Thread Modes

My python code is running very slow on millions of records

shantanu97
Silly Frenchman

Posts: 38

Threads: 31

Joined: Feb 2021

Reputation: 0

Dec-28-2021, 12:50 AM

(Dec-27-2021, 12:22 PM)paul18fr Wrote: What type of data are you dealing with in the original csv file? pure numbers? strings? both? The

Appending is costly, and maybe loops can be avoided using vectorisation if data are numbers.

It consists of a string, number and a date.

Find

Messages In This Thread

My python code is running very slow on millions of records - by shantanu97 - Dec-27-2021, 11:02 AM

RE: My python code is running very slow on millions of records - by paul18fr - Dec-27-2021, 12:22 PM

RE: My python code is running very slow on millions of records - by shantanu97 - Dec-28-2021, 12:50 AM

RE: My python code is running very slow on millions of records - by Larz60+ - Dec-27-2021, 11:18 PM

RE: My python code is running very slow on millions of records - by shantanu97 - Dec-28-2021, 12:54 AM

RE: My python code is running very slow on millions of records - by Larz60+ - Dec-28-2021, 02:23 AM

RE: My python code is running very slow on millions of records - by shantanu97 - Dec-28-2021, 02:34 AM

RE: My python code is running very slow on millions of records - by Larz60+ - Dec-28-2021, 11:02 AM

Possibly Related Threads…
Thread		Author	Replies	Views	Last Post
	writing and running code in vscode without saving it	akbarza	5	2,228	Mar-03-2025, 08:14 PM Last Post: Gribouillis
	Python: How to import data from txt, instead of running the data from the code?	Melcu54	1	600	Dec-13-2024, 06:50 AM Last Post: Gribouillis
	Why the python is so slow?	rohhthone	3	1,066	Oct-07-2024, 09:54 PM Last Post: DeaD_EyE
	Sudden Extremely Slow / Failed Python Imports	bmccollum	1	1,082	Aug-20-2024, 02:09 PM Last Post: DeaD_EyE
	problem in running a code	akbarza	7	2,240	Feb-14-2024, 02:57 PM Last Post: snippsat
	the order of running code in a decorator function	akbarza	2	1,288	Nov-10-2023, 08:09 AM Last Post: akbarza
	validate large json file with millions of records in batches	herobpv	3	2,140	Dec-10-2022, 10:36 PM Last Post: bowlofred
	How to retrieve records in a DataFrame (Python/Pandas) that contains leading or trail	mmunozjr	3	3,511	Sep-05-2022, 11:56 AM Last Post: Pedroski55
	Code running many times nad not just one?	korenron	4	2,155	Jul-24-2022, 08:12 AM Last Post: korenron
	Error while running code on VSC	maiya	4	6,608	Jul-01-2022, 02:51 PM Last Post: maiya

Users browsing this thread: 1 Guest(s)

View a Printable Version

My python code is running very slow on millions of records

User Panel Messages

Announcements