Python Forum
How to get memory usage and execution time of each line in python
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to get memory usage and execution time of each line in python
#1
Hi,
I have below code, and I want to get(calculate) the memory usage and execution time of each line of my code. My code is below, kindly someone help,

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, cbar_kws={'ticks': [0, 2, 4, 6, 8, 10]}, vmin=0, vmax=10) 
plt.show()

 
#Create a DataFrame
d = {'Name':['Alisa','Bobby','Cathrine','Madonna','Rocky','Sebastian','Jaqluine',
   'Rahul','David','Andrew','Ajay','Teresa'],
   'Score1':[62,47,55,74,31,77,85,63,42,32,71,57],
   'Score2':[89,87,67,55,47,72,76,79,44,92,99,69]}
 
 
df = pd.DataFrame(d)
col_mean=df.mean()
col_std=df.std()
get_disc=df.describe()
Reply
#2
For memory usage there is built-in gc - Garbage Collector interface. With gc module one can directly interact with the garbage collector and figure out what objects it's tracking as references and how many of those objects there are.

Starting from Python 3.4 there is also built-in tracemalloc - Trace memory allocations which makes possible to connect an object back to where it was allocated.

For performance measurement there are profilers.
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy

Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Reply
#3
I go through those stuff, due to I am very new to python, I could not understand how to arrange my code. However I do the following, but it does not give any memory usage summary

I refer to: https://pypi.org/project/memory-profiler/

from memory_profiler import profile
@profile
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()
uniform_data = np.random.rand(10, 12)
ax = sns.heatmap(uniform_data, cbar_kws={'ticks': [0, 2, 4, 6, 8, 10]}, vmin=0, vmax=10) 
plt.show()
 
  
#Create a DataFrame
d = {'Name':['Alisa','Bobby','Cathrine','Madonna','Rocky','Sebastian','Jaqluine',
   'Rahul','David','Andrew','Ajay','Teresa'],
   'Score1':[62,47,55,74,31,77,85,63,42,32,71,57],
   'Score2':[89,87,67,55,47,72,76,79,44,92,99,69]}
  
  
df = pd.DataFrame(d)
col_mean=df.mean()
col_std=df.std()
get_disc=df.describe()
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  python memory rd_rakesh 1 243 Jul-14-2021, 11:15 AM
Last Post: jefsummers
  How to measure execution time of a multithread loop spacedog 2 527 Apr-24-2021, 07:52 AM
Last Post: spacedog
  PyCharm Script Execution Time? muzikman 3 1,489 Dec-14-2020, 11:22 PM
Last Post: muzikman
  Usage of format() cheers100 2 466 Nov-13-2020, 01:14 PM
Last Post: cheers100
  Windows Python Memory Scanner Awesometech 1 4,612 Oct-14-2020, 07:44 AM
Last Post: badengagen
  Print characters in a single line rather than one at a time hhydration 1 612 Oct-10-2020, 10:00 PM
Last Post: bowlofred
  memory leak on embedded python in c++ asdf3721 3 1,003 Jul-16-2020, 06:33 AM
Last Post: Gribouillis
  How to to tie the execution of one process to another inside a loop in Python ignorant_wanderer 0 603 Jul-11-2020, 03:44 AM
Last Post: ignorant_wanderer
  Python - change variable type during program execution ple 1 800 Apr-12-2020, 08:43 AM
Last Post: buran
  Memory consumption grows during execution only on single machine Jendker 2 734 Feb-10-2020, 01:57 PM
Last Post: Jendker

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020