Python Forum
How to get memory usage and execution time of each line in python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to get memory usage and execution time of each line in python
#1
Hi,
I have below code, and I want to get(calculate) the memory usage and execution time of each line of my code. My code is below, kindly someone help,

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, cbar_kws={'ticks': [0, 2, 4, 6, 8, 10]}, vmin=0, vmax=10) 
plt.show()

 
#Create a DataFrame
d = {'Name':['Alisa','Bobby','Cathrine','Madonna','Rocky','Sebastian','Jaqluine',
   'Rahul','David','Andrew','Ajay','Teresa'],
   'Score1':[62,47,55,74,31,77,85,63,42,32,71,57],
   'Score2':[89,87,67,55,47,72,76,79,44,92,99,69]}
 
 
df = pd.DataFrame(d)
col_mean=df.mean()
col_std=df.std()
get_disc=df.describe()
Reply
#2
For memory usage there is built-in gc - Garbage Collector interface. With gc module one can directly interact with the garbage collector and figure out what objects it's tracking as references and how many of those objects there are.

Starting from Python 3.4 there is also built-in tracemalloc - Trace memory allocations which makes possible to connect an object back to where it was allocated.

For performance measurement there are profilers.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#3
I go through those stuff, due to I am very new to python, I could not understand how to arrange my code. However I do the following, but it does not give any memory usage summary

I refer to: https://pypi.org/project/memory-profiler/

from memory_profiler import profile
@profile
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, cbar_kws={'ticks': [0, 2, 4, 6, 8, 10]}, vmin=0, vmax=10) 
plt.show()
 
  
#Create a DataFrame
d = {'Name':['Alisa','Bobby','Cathrine','Madonna','Rocky','Sebastian','Jaqluine',
   'Rahul','David','Andrew','Ajay','Teresa'],
   'Score1':[62,47,55,74,31,77,85,63,42,32,71,57],
   'Score2':[89,87,67,55,47,72,76,79,44,92,99,69]}
  
  
df = pd.DataFrame(d)
col_mean=df.mean()
col_std=df.std()
get_disc=df.describe()
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  kill python execution program lebossejames 0 223 Mar-16-2024, 11:16 AM
Last Post: lebossejames
  Line by line execution of instructions JosephR 3 689 Jul-24-2023, 09:10 PM
Last Post: deanhystad
  python list out top 10 Memory utilisation mg24 0 798 Nov-20-2022, 10:51 PM
Last Post: mg24
  Confused about python execution jpezz 4 1,367 Oct-09-2022, 06:56 PM
Last Post: Gribouillis
Bug Embedded Python Memory Leaks Alexei 1 1,020 Sep-16-2022, 01:15 PM
Last Post: Alexei
  How to change UTC time to local time in Python DataFrame? SamKnight 2 1,587 Jul-28-2022, 08:23 AM
Last Post: Pedroski55
  multi-line CMD in one-line python kucingkembar 5 3,942 Jan-01-2022, 12:45 PM
Last Post: kucingkembar
  Reducing runtime memory usage in Cpython interpreter david_the_graower 2 2,214 Oct-18-2021, 09:56 PM
Last Post: david_the_graower
  python memory rd_rakesh 1 1,640 Jul-14-2021, 11:15 AM
Last Post: jefsummers
  How to measure execution time of a multithread loop spacedog 2 2,859 Apr-24-2021, 07:52 AM
Last Post: spacedog

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020