Python Forum
[Tkinter] acceleration of data output in treeview tkinter
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
[Tkinter] acceleration of data output in treeview tkinter
#1
Good afternoon!

I am trying to come up with a mechanism to output data from the database in treeview tkinter (tabular form).
I read the table from the database, divide the number of rows by the number of processor cores.
I create streams and try to write data in parallel to treeview via write function.

def trvwinsert(lstb_donnes, numstrt, numfnsh):
    brak = 0
    colbrak = []
    goden = 0
    colgoden = []

    for n in range(numstrt, numfnsh):
        if lstb_donnes[n][6] == 'defect':
            listBox.insert("", "end", values=(lstb_donnes[n][:]), tags=('defect',))
            brak = brak + 1
        else:
            listBox.insert("", "end", values=(lstb_donnes[n][:]))
            goden = goden + 1
    return colgoden.append(goden), colbrak.append(brak)
Function code with dialog box (top level):

# mlticpu = multiprocessing.cpu_count()
        lenprc = len(lstb_donnes)

        nstrt = []
        nfnsh = []
        thread_list = []

        for j in range(mlticpu):
            if j == 0:
                nstrt.append(0)
            else:
                nstrt.append(nfnsh[j - 1] + 1)

            if j == mlticpu - 1:
                nfnsh.append(lenprc)
            else:
                nfnsh.append(nstrt[j] + math.ceil(lenprc / mlticpu))

        start = time.time()

        for h in range(mlticpu):
            numstrt = nstrt[h]
            numfnsh = nfnsh[h]
            t = threading.Thread(target=trvwinsert, name='thread{}'.format(h), args=(lstb_donnes, numstrt, numfnsh))
            thread_list.append(t)
            t.start()
            print('{} has started'.format(t.name))

        for t in thread_list:
            t.join()

        end = time.time()

        print("cycle time = " + str(end - start))
        print('all threads done')

        first_item.entryconfig(1, label="Сохранить отчет", state="normal")
        dop_window.after(1000)
        dop_window.destroy()
As a result of execution, even with one thread, I get an error:

Exception in thread thread0:
Traceback (most recent call last):
  File "C:\Python\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "C:\Python\lib\threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "C:/TestInstruments/Reports/report_4_new variant.py", line 68, in trvwinsert
    listBox.insert("", "end", values=(lstb_donnes[n][:]))
  File "C:\Python\lib\tkinter\ttk.py", line 1369, in insert
    res = self.tk.call(self._w, "insert", parent, index, *opts)
RuntimeError: main thread is not in main loop
As a result of searching for a solution to the problem, I did not come to a consensus on whether it is possible to speed up the writing of data to treeview using the threading module. It may be necessary to use the multiprocessing module.
The main question is whether it is possible to speed up the output of large tables in treeview tkinter. Now the output takes 40-45 minutes and only loads one core of the CPU.
Reply
#2
Quote:Now the output takes 40-45 minutes and only loads one core of the CPU
How many rows of data are involved?
This seems like it might be an SQL query issue more than tkinter (treeview in all of my experience has been fast)

Have you tried running your query without any GUI overhead?
Can you share the SQL?
Reply
#3
Good afternoon!

There are a million lines of data to output to such a table, maybe more. I first read the data from the database already prepared for display on the screen. I bring them out of memory and not from the base. The output line by line to the table takes time, if it is 5 thousand lines then it is seconds, but if 500 000 and more it is minutes.

I wanted to write data in multiple lines from multiple streams to a parallel.

I read the sources on this topic, I realized that I was mistaken - I had to build an interface on PySide2 (PyQT5). If necessary, I will redo it (tables are better implemented in it and output multithreading is supported).
Reply
#4
I think you should try PYQT5 or (my preference) wxpython.
Both are much newer code than tkinter and probably (I don't know this for a fact) faster.

I don't know that multithreading would help, can't wrap my mind around how you would synchronize the writing to the widget, as I'm pretty sure there's no way to write more than one row at a time.

You're just putting the data key in the treeview, correct?
the actual data should be in memory and accessed by the key, then displayed in some other widget.
Tkinter is not the best choice for this type of display as the widget set is quite limited.
Reply
#5
I think the topic can be closed or moved to a solved one.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Displaying output in GUI ( Tkinter) Zouloutamtam 7 276 Sep-29-2020, 02:08 PM
Last Post: Zouloutamtam
  [PyGUI] Expand last sub-node of treeview widget mart79 1 397 Apr-25-2020, 07:57 PM
Last Post: mart79
  Treeview - How to lock columns taratata2020 0 448 Mar-29-2020, 03:30 PM
Last Post: taratata2020
  [Tkinter] Treeview expand button size shamnadinn 8 1,247 Mar-08-2020, 12:02 PM
Last Post: shamnadinn
  Active tkinter text output during loop dvanommen 2 3,036 Oct-18-2019, 02:23 PM
Last Post: dvanommen
  [PyQt] How to populate a treeview on a GUI with a dictionary mart79 1 2,311 Aug-05-2019, 01:30 PM
Last Post: Denni
  TKinter GUI / Pandas Data frame Loop NSearch 0 3,561 Jun-26-2019, 12:37 AM
Last Post: NSearch
  [Tkinter] Not getting entry values on button click & treeview not updating ? swanysto 4 2,964 May-10-2019, 04:16 PM
Last Post: swanysto
  [Tkinter] Image does not show in treeview. KevinBrown 3 1,927 May-05-2019, 11:47 PM
Last Post: KevinBrown
  [Tkinter] Tkinter timetabl using treeview mntfr 3 928 Feb-05-2019, 09:36 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020