Python Forum

Hi everyone. I have a python script (for testing). Its simply format a string:

import time

class Profiler(object):
    def __enter__(self):
        self._startTime = time.time()

    def __exit__(self, type, value, traceback):
        print("Elapsed time: {:.3f} sec".format(time.time() - self._startTime))

def cycle():
    query = ''
    elem = {'sec_code' : "test", 'face_unit' : "test", 'class_code' : "test", 'code' : "test", 'scale' : 'test', 'face_value':'test',
            'lot_size': "test", 'short_name' : 'test', 'name': "test", 'min_price_step' : 'test', 'isin_code': 'test', 'class_name' : 'test', 'mat_date': 'test'}
    for i  in range(45000):

        query = query + "insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name," \
                        "name,min_price_step,isin_code,class_name,mat_date)" \
                        "values ('{}', '{}', '{}', '{}', {}, {}, {}, '{}', '{}', {}, '{}', '{}', '{}')  ON CONFLICT DO NOTHING;".format(
            elem['sec_code'],
            elem['face_unit'],
            elem['class_code'],
            elem['code'],
            elem['scale'],
            elem['face_value'],
            elem['lot_size'],
            elem['short_name'].replace("'", ""),
            elem['name'].replace("'", ""),
            elem['min_price_step'],
            elem['isin_code'],
            elem['class_name'], elem['mat_date'])

    #print (''.join(query))

if __name__ == "__main__":
    with Profiler() as p:
        cycle()

On my notebook Asus K73E CPU intel i5-2340M (2 cores 2,40 GHz) 8GB RAM this script executes for 9.8 seconds. I thought it is too much and buy desctop PC with much performance: AMD Ryzen 5 2600 (6 cores 3.4GHz) and 18GB RAM PC-24300. Run on it same script and was shocked: script executes for 8.6 seconds.
I do not ask about optimization of the script, use of multithreading, etc. (formation of an object of list and append will work less than a second). Interests me why time of execution is so big in the form in which it is. Because two persons from a stackoverflow forum wrote that at them this script was executed for 50-60 milliseconds. For 60 milliseconds there was time on five years old laptop. So i cant understand why 8 seconds for me. I tried:
different python versions (32 bit and 64 bit) (2.7, 3.7, 3.8) 3.8 was faster = 7.5 sec.
differet OS: win7 and win10 64 bit
launch from PyCharm and from command line - results are same

There are no ideas where to look for a problem

I noticed also that if to increase the size of a string line a little, then performance time also grows by seconds.

I got

Output:
Elapsed time: 0.056 sec

My platform is Kubuntu linux with a intel Core i7-6700 processor at 3.4 Ghz.

import datetime

def cycle():

    query = ''
    
    print (datetime.datetime.now())
    for i  in range(45000):
        query = query + "insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit"
    print (datetime.datetime.now())
if __name__ == "__main__":

    cycle()

Easier code on Linux machine Python 2.6 executes for few milliseconds (hardware much lower). On Windows7 machine same code runs for 43 seconds. Where look for problem????

You could perhaps profile memory usage with the tracemalloc module (standard library). There are other online tools for this too. You could compare the results in Linux and Windows.

CProfile output:

Quote:test2.py:11: size=10.7 MiB, count=1, average=10.7 MiB
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:522: size=199 B, count=4, average=50 B
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:519: size=197 B, count=4, average=49 B
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:515: size=96 B, count=2, average=48 B
test2.py:10: size=16 B, count=1, average=16 B
6231 function calls (6125 primitive calls) in 7.212 seconds

Ordered by: internal time

ncalls tottime percall cumtime percall filename:lineno(function)
1 7.196 7.196 7.197 7.197 test2.py:4(cycle)
63 0.003 0.000 0.003 0.000 {built-in method nt.stat}
9 0.002 0.000 0.002 0.000 {built-in method marshal.loads}
1 0.001 0.001 7.212 7.212 test2.py:1(<module>)
9 0.001 0.000 0.001 0.000 <frozen importlib._bootstrap_external>:830(get_data)
29 0.001 0.000 0.001 0.000 {built-in method builtins.__build_class__}
45 0.001 0.000 0.004 0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
11/1 0.000 0.000 7.212 7.212 {built-in method builtins.exec}
6 0.000 0.000 0.001 0.000 {built-in method builtins.print}

What is built-in method builtins.exec?