Python Forum
[Tkinter] acceleration of data output in treeview tkinter
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[Tkinter] acceleration of data output in treeview tkinter
#1
Good afternoon!

I am trying to come up with a mechanism to output data from the database in treeview tkinter (tabular form).
I read the table from the database, divide the number of rows by the number of processor cores.
I create streams and try to write data in parallel to treeview via write function.

def trvwinsert(lstb_donnes, numstrt, numfnsh):
    brak = 0
    colbrak = []
    goden = 0
    colgoden = []

    for n in range(numstrt, numfnsh):
        if lstb_donnes[n][6] == 'defect':
            listBox.insert("", "end", values=(lstb_donnes[n][:]), tags=('defect',))
            brak = brak + 1
        else:
            listBox.insert("", "end", values=(lstb_donnes[n][:]))
            goden = goden + 1
    return colgoden.append(goden), colbrak.append(brak)
Function code with dialog box (top level):

# mlticpu = multiprocessing.cpu_count()
        lenprc = len(lstb_donnes)

        nstrt = []
        nfnsh = []
        thread_list = []

        for j in range(mlticpu):
            if j == 0:
                nstrt.append(0)
            else:
                nstrt.append(nfnsh[j - 1] + 1)

            if j == mlticpu - 1:
                nfnsh.append(lenprc)
            else:
                nfnsh.append(nstrt[j] + math.ceil(lenprc / mlticpu))

        start = time.time()

        for h in range(mlticpu):
            numstrt = nstrt[h]
            numfnsh = nfnsh[h]
            t = threading.Thread(target=trvwinsert, name='thread{}'.format(h), args=(lstb_donnes, numstrt, numfnsh))
            thread_list.append(t)
            t.start()
            print('{} has started'.format(t.name))

        for t in thread_list:
            t.join()

        end = time.time()

        print("cycle time = " + str(end - start))
        print('all threads done')

        first_item.entryconfig(1, label="Сохранить отчет", state="normal")
        dop_window.after(1000)
        dop_window.destroy()
As a result of execution, even with one thread, I get an error:

Exception in thread thread0:
Traceback (most recent call last):
  File "C:\Python\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Python\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:/TestInstruments/Reports/report_4_new variant.py", line 68, in trvwinsert
    listBox.insert("", "end", values=(lstb_donnes[n][:]))
  File "C:\Python\lib\tkinter\ttk.py", line 1369, in insert
    res = self.tk.call(self._w, "insert", parent, index, *opts)
RuntimeError: main thread is not in main loop
As a result of searching for a solution to the problem, I did not come to a consensus on whether it is possible to speed up the writing of data to treeview using the threading module. It may be necessary to use the multiprocessing module.
The main question is whether it is possible to speed up the output of large tables in treeview tkinter. Now the output takes 40-45 minutes and only loads one core of the CPU.
Reply
#2
Quote:Now the output takes 40-45 minutes and only loads one core of the CPU
How many rows of data are involved?
This seems like it might be an SQL query issue more than tkinter (treeview in all of my experience has been fast)

Have you tried running your query without any GUI overhead?
Can you share the SQL?
Reply
#3
Good afternoon!

There are a million lines of data to output to such a table, maybe more. I first read the data from the database already prepared for display on the screen. I bring them out of memory and not from the base. The output line by line to the table takes time, if it is 5 thousand lines then it is seconds, but if 500 000 and more it is minutes.

I wanted to write data in multiple lines from multiple streams to a parallel.

I read the sources on this topic, I realized that I was mistaken - I had to build an interface on PySide2 (PyQT5). If necessary, I will redo it (tables are better implemented in it and output multithreading is supported).
Reply
#4
I think you should try PYQT5 or (my preference) wxpython.
Both are much newer code than tkinter and probably (I don't know this for a fact) faster.

I don't know that multithreading would help, can't wrap my mind around how you would synchronize the writing to the widget, as I'm pretty sure there's no way to write more than one row at a time.

You're just putting the data key in the treeview, correct?
the actual data should be in memory and accessed by the key, then displayed in some other widget.
Tkinter is not the best choice for this type of display as the widget set is quite limited.
Reply
#5
I think the topic can be closed or moved to a solved one.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  [Tkinter] [split] Is there a way to embed a treeview as a row inside another treeview? CyKlop 5 203 Oct-20-2021, 12:14 AM
Last Post: CyKlop
  Tkinter | entry output. Sap2ch 1 209 Sep-25-2021, 12:38 AM
Last Post: Yoriz
  tkinter - update/refresh treeview snakes 4 3,433 May-13-2021, 07:10 AM
Last Post: snakes
  Displaying output in GUI ( Tkinter) Zouloutamtam 7 5,899 Sep-29-2020, 02:08 PM
Last Post: Zouloutamtam
  Active tkinter text output during loop dvanommen 2 6,288 Oct-18-2019, 02:23 PM
Last Post: dvanommen
  [Tkinter] Tkinter timetabl using treeview mntfr 3 1,626 Feb-05-2019, 09:36 PM
Last Post: Larz60+
  sQlite3 output to tkinter treeview - how do I set / increase width of the output? dewijones67 5 4,056 Jan-23-2019, 08:45 AM
Last Post: Larz60+
  [Tkinter] I need to know how to put my SQLite3 data into a Treeview Thename921 1 4,378 Jan-12-2019, 10:26 PM
Last Post: Larz60+
  [Tkinter] tkinter - unexpected output - don't know how func is triggered before calling omm 8 2,141 Dec-11-2018, 08:09 PM
Last Post: Gribouillis
  [Tkinter] Tkinter widgets driving Arduino uno output pins woodcncwnc 3 2,689 Jan-29-2018, 08:21 PM
Last Post: woodcncwnc

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020