Python performance

[rvkstudent]

Hi everyone. I have a python script (for testing). Its simply format a string:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36

import time   class Profiler(object):     def __enter__(self):         self._startTime = time.time()       def __exit__(self, type, value, traceback):         print("Elapsed time: {:.3f} sec".format(time.time() - self._startTime))   def cycle():     query = ''     elem = {'sec_code' : "test", 'face_unit' : "test", 'class_code' : "test", 'code' : "test", 'scale' : 'test', 'face_value':'test',             'lot_size': "test", 'short_name' : 'test', 'name': "test", 'min_price_step' : 'test', 'isin_code': 'test', 'class_name' : 'test', 'mat_date': 'test'}     for i  in range(45000):           query = query + "insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name," \                         "name,min_price_step,isin_code,class_name,mat_date)" \                         "values ('{}', '{}', '{}', '{}', {}, {}, {}, '{}', '{}', {}, '{}', '{}', '{}')  ON CONFLICT DO NOTHING;".format(             elem['sec_code'],             elem['face_unit'],             elem['class_code'],             elem['code'],             elem['scale'],             elem['face_value'],             elem['lot_size'],             elem['short_name'].replace("'", ""),             elem['name'].replace("'", ""),             elem['min_price_step'],             elem['isin_code'],             elem['class_name'], elem['mat_date'])       #print (''.join(query))   if __name__ == "__main__":     with Profiler() as p:         cycle()

On my notebook Asus K73E CPU intel i5-2340M (2 cores 2,40 GHz) 8GB RAM this script executes for 9.8 seconds. I thought it is too much and buy desctop PC with much performance: AMD Ryzen 5 2600 (6 cores 3.4GHz) and 18GB RAM PC-24300. Run on it same script and was shocked: script executes for 8.6 seconds.
I do not ask about optimization of the script, use of multithreading, etc. (formation of an object of list and append will work less than a second). Interests me why time of execution is so big in the form in which it is. Because two persons from a stackoverflow forum wrote that at them this script was executed for 50-60 milliseconds. For 60 milliseconds there was time on five years old laptop. So i cant understand why 8 seconds for me. I tried:
different python versions (32 bit and 64 bit) (2.7, 3.7, 3.8) 3.8 was faster = 7.5 sec.
differet OS: win7 and win10 64 bit
launch from PyCharm and from command line - results are same

There are no ideas where to look for a problem

I noticed also that if to increase the size of a string line a little, then performance time also grows by seconds.

[Gribouillis]

I got

Output:
Elapsed time: 0.056 sec

My platform is Kubuntu linux with a intel Core i7-6700 processor at 3.4 Ghz.

[rvkstudent]

1 2 3 4 5 6 7 8 9 10 11 12 13

import datetime   def cycle():       query = ''           print (datetime.datetime.now())     for i  in range(45000):         query = query + "insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit insert into securyties(sec_code, face_unit, class_code,code, scale, face_value, lot_size,short_name values insert into securyties(sec_code, face_unit"     print (datetime.datetime.now()) if __name__ == "__main__":       cycle()

Easier code on Linux machine Python 2.6 executes for few milliseconds (hardware much lower). On Windows7 machine same code runs for 43 seconds. Where look for problem????

[Gribouillis]

You could perhaps profile memory usage with the tracemalloc module (standard library). There are other online tools for this too. You could compare the results in Linux and Windows.

[rvkstudent]

CProfile output:

test2.py:11: size=10.7 MiB, count=1, average=10.7 MiB
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:522: size=199 B, count=4, average=50 B
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:519: size=197 B, count=4, average=49 B
C:\Users\kozlov.r\AppData\Local\Programs\Python\Python36-32\lib\tracemalloc.py:515: size=96 B, count=2, average=48 B
test2.py:10: size=16 B, count=1, average=16 B
6231 function calls (6125 primitive calls) in 7.212 seconds

Ordered by: internal time

ncalls tottime percall cumtime percall filename:lineno(function)
1 7.196 7.196 7.197 7.197 test2.py:4(cycle)
63 0.003 0.000 0.003 0.000 {built-in method nt.stat}
9 0.002 0.000 0.002 0.000 {built-in method marshal.loads}
1 0.001 0.001 7.212 7.212 test2.py:1(<module>)
9 0.001 0.000 0.001 0.000 <frozen importlib._bootstrap_external>:830(get_data)
29 0.001 0.000 0.001 0.000 {built-in method builtins.__build_class__}
45 0.001 0.000 0.004 0.000 <frozen importlib._bootstrap_external>:1233(find_spec)
11/1 0.000 0.000 7.212 7.212 {built-in method builtins.exec}
6 0.000 0.000 0.001 0.000 {built-in method builtins.print}

What is built-in method builtins.exec?