Python Forum
Arrays faster than pandas?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Arrays faster than pandas?
#1
I just learned that Python is slow with regard to numbers because they are generally wrapped as objects with associated attributes. The memory burden can therefore be 100x what it might take to store the raw number itself. According to the video, NumPy is so fast because it optimizes numerical encoding (e.g. three bits for the number seven as 100 in binary) and because everything is processed as arrays (and therefore stored in contiguous memory locations).

Assuming this is all correct, I wonder: are pandas dataframes stored as arrays in the same manner? If not, then would it be faster to try and store the data in arrays and process them through numpy rather than in dataframes processed through pandas?

Thanks!
Reply


Messages In This Thread
Arrays faster than pandas? - by Mark17 - Jul-26-2021, 06:07 PM
RE: Arrays faster than pandas? - by jefsummers - Jul-27-2021, 04:01 PM
RE: Arrays faster than pandas? - by Mark17 - Jul-31-2021, 12:37 PM
RE: Arrays faster than pandas? - by jefsummers - Jul-31-2021, 09:00 PM
RE: Arrays faster than pandas? - by Mark17 - Aug-02-2021, 03:14 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas dataframes and numpy arrays bytecrunch 1 1,430 Oct-11-2022, 08:08 PM
Last Post: Larz60+
  comparing floating point arrays to arrays of integers in Numpy amjass12 0 1,705 Jul-26-2021, 11:58 AM
Last Post: amjass12
  Numpy arrays and compatability with Fortran arrays merrittr 0 1,955 Sep-03-2019, 03:54 AM
Last Post: merrittr
  Pandas: faster method to count occurrences frame 0 2,373 May-26-2019, 07:45 PM
Last Post: frame

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020