Looking for clarification related to performance

[casevh]

It is difficult to provide a complete answer without understanding the context - both of the person making the statement "function calls are slow" and what your program does. So let's look at adding two numbers that are stored in the variables a and b. The easiest way is just t = a + b.

It is relatively fast.

$ python3.9 -m timeit -s "a=1;b=2" "t=a+b"
10000000 loops, best of 5: 20.7 nsec per loop

Now create a function and save it in a file called examples.py

def f_1(x, y):
    return x + y

Now measure the running time.

$ python3.9 -m timeit -s "import examples;a=1;b=1" "examples.f_1(a,b)"
5000000 loops, best of 5: 94.8 nsec per loop

Why is it so much slower? One reason is that Python uses dictionaries to store the names of functions. There are two additional dictionary lookups required to evaluate examples.f_1(a,b). Python first must find examples in the local namespace. Then it must find f_1 in the local namespace belonging to examples. A common trick is to lookup examples.f_1 once and cache the result. For example f=examples.f_1. That is faster.

$ python3.9 -m timeit -s "import examples;f=examples.f_1;a=1;b=1" "f(a,b)"
5000000 loops, best of 5: 70.2 nsec per loop

So there is an overhead for calling a function. So does that imply we minimize the use of functions? Nope. Surprisingly, code inside a function can run faster. Here is a contrived example. The code:

import time

big_list = [(i,i) for i in range(500)]

start = time.time()
total = 0
for a,b in big_list:
    total = total + a * b
end = time.time()
print("Elapsed time:", end - start)

def f_2(big_list):
    start = time.time()
    total = 0
    for a,b in big_list:
        total = total + a * b
    end = time.time()
    print("Elapsed time:", end - start)

f_2(big_list)

and the results:

$ python3.9 example2.py 
Elapsed time: 6.580352783203125e-05
Elapsed time: 3.337860107421875e-05

Why the difference? Name lookups that are in a function's own local namespace are faster than a dictionary lookup.

---

So what to do? Organize code as functions just because it is easier to write, understand, and maintain. If the program is running too slow, look carefully at "how" you are solving your problem. Better algorithms are, well, better. Identify what code is slow and understand why it slow. Learn itertools (see next section for why).

---

I added the following function to example2.

import itertools
import operator
def f_3(big_list):
    start = time.time()
    sum((itertools.starmap(operator.mul,big_list)))
    end = time.time()
    print("Elapsed time:", end - start)

f_3(big_list)

and the results:

$ python3.9 example2.py 
Elapsed time: 6.222724914550781e-05
Elapsed time: 3.314018249511719e-05
Elapsed time: 1.9311904907226562e-05

---

Lastly, there are many performance recommendations on the Internet. Don't trust their recommendations unless you measure it on _your_ code. (Don't trust me either. I chose my examples to illustrate specific behaviors.)

p.s. The fastest way to compute the sum of squares is the formula (n(n+1)(2n+1)) / 6. A better algorithm is, well, better.

casevh