Waiting for heavy functions question

[philipbergwerf]

I have written a program that writes a music score based on a text file with a certain syntax. I use tkinter as gui and draw engine to draw the score and be able to export the score to pdf. It was much work making all this so I don't want to rewrite everything in a different way to solve the following problem:
The program does this cycle; everytime you release a key in the tkinter text widget it runs the render() function and the score gets updated. When having a few notes in the score this works really quick but when it has to render more than 400 notes every render starts to delay the text input. So only when I am quickly typing in a big score, the text widget waits for the render() function to complete before updating it's contents.
I made a little example:

from tkinter import *

root = Tk()

canvas = Canvas(root)
canvas.pack()

text = Text(root)
text.pack()

def render(event=None):
	'''
	This functions represents a really
	heavy render function that can render
	a lot of pages and turn into the problem
	that the function slows down the user 
	experience.
	'''
	canvas.create_line(10,10,100,100)

'''
we bind the KeyRelease. When we enter quickly
multiple characters to the text box and the render()
function has to render 10 pages of music, the text 
widget is going to wait for the render function to 
complete before updating the text widget which is
anoying for the user.
'''
root.bind('<KeyRelease>', render)

root.mainloop()

How can we program the above example in such a way that the text widget is never waiting for the render function to complete before updating itself? Typing in the text widget should always be a fluent experience. It doesn't matter if the render function takes time to complete but it shouldn't slow down the text widget.

[Gribouillis]

In the past, I had some success using this activestate recipe for long running tasks in tkinter. While already five years old, it should still work.

Note that this will start a thread when tk_call_async() is called. You may want to organize your code to avoid starting a new thread for every keystroke.

[deanhystad]

It might be easiest to use a two step operation. Type then Apply. You could even do this without any changes to your code other than binding <Return> to call your render method.

[philipbergwerf]

I made a solution that works really wel. In short it works like this:

from tkinter import *
import time
from threading import Thread

root = Tk()

text = Text(root)
text.pack()

program_is_running = True
render_needs_to_render = True

def render(event=None):
    '''
    This functions represents a really
    heavy render function that can render
    a lot of pages.
    '''
    print('render')

'''
This function is running in a thread.
Every 0.5 seconds it's checking
if it needs to run the main render
function.
'''
def thread_auto_render():
    global render_needs_to_render
    while program_is_running == True:
        if render_needs_to_render:
            render_needs_to_render = False
            render()
        time.sleep(0.5)
Thread(target=thread_auto_render).start()


'''
If we close the program program_is_running
will turn False so the render thread is ending
'''
def quit_program(event):
    global program_is_running
    program_is_running = False
    time.sleep(0.51)
    root.destroy()

'''
it only needs to render when we type something
in the text box.
'''
def on_keyrelease(event):
    global render_needs_to_render
    render_needs_to_render = True

text.bind('<KeyRelease>', render)
root.bind('<Escape>', quit_program)

root.mainloop()

[Gribouillis]

I think you could improve this by using a Threading.Condition instance so that instead of a periodic loop, the thread simply sits waiting for user input which is indicated by notifying the Condition.

from tkinter import *
import time
from threading import Condition, Thread
 
root = Tk()
 
text = Text(root)
text.pack()
 
program_is_running = True
 
def render(event=None):
    '''
    This functions represents a really
    heavy render function that can render
    a lot of pages.
    '''
    print('render')

class QuitThread(Exception):
    pass

class ThreadAutoRender(Thread):
    def __init__(self):
        Thread.__init__(self)
        self.needs_to_render = False
        self.condition = Condition()
    def run(self):
        try:
            while True:
                self.wait_and_render()
        except QuitThread:
            pass
        
    def wait_and_render(self):
        with self.condition:
            if not program_is_running:
                raise QuitThread
            while not self.needs_to_render:
                self.condition.wait()
                if not program_is_running:
                    raise QuitThread
            self.needs_to_render = False
        render()

thread_auto_render = ThreadAutoRender()
thread_auto_render.start()

'''
If we close the program program_is_running
will turn False so the render thread is ending
'''
def quit_program(event):
    global program_is_running
    with thread_auto_render.condition:
        program_is_running = False
        thread_auto_render.condition.notify()
    thread_auto_render.join()
    root.destroy()
 
'''
it only needs to render when we type something
in the text box.
'''
def on_keyrelease(event):
    with thread_auto_render.condition:
        thread_auto_render.needs_to_render = True
        thread_auto_render.condition.notify()
 
text.bind('<KeyRelease>', on_keyrelease)
root.bind('<Escape>', quit_program)
 
root.mainloop()

[philipbergwerf]

I think you could improve this by using a Threading.Condition instance so that instead of a periodic loop, the thread simply sits waiting for user input which is indicated by notifying the Condition.

from tkinter import *
import time
from threading import Condition, Thread
 
root = Tk()
 
text = Text(root)
text.pack()
 
program_is_running = True
 
def render(event=None):
    '''
    This functions represents a really
    heavy render function that can render
    a lot of pages.
    '''
    print('render')

class QuitThread(Exception):
    pass

class ThreadAutoRender(Thread):
    def __init__(self):
        Thread.__init__(self)
        self.needs_to_render = False
        self.condition = Condition()
    def run(self):
        try:
            while True:
                self.wait_and_render()
        except QuitThread:
            pass
        
    def wait_and_render(self):
        with self.condition:
            if not program_is_running:
                raise QuitThread
            while not self.needs_to_render:
                self.condition.wait()
                if not program_is_running:
                    raise QuitThread
            self.needs_to_render = False
        render()

thread_auto_render = ThreadAutoRender()
thread_auto_render.start()

'''
If we close the program program_is_running
will turn False so the render thread is ending
'''
def quit_program(event):
    global program_is_running
    with thread_auto_render.condition:
        program_is_running = False
        thread_auto_render.condition.notify()
    thread_auto_render.join()
    root.destroy()
 
'''
it only needs to render when we type something
in the text box.
'''
def on_keyrelease(event):
    with thread_auto_render.condition:
        thread_auto_render.needs_to_render = True
        thread_auto_render.condition.notify()
 
text.bind('<KeyRelease>', on_keyrelease)
root.bind('<Escape>', quit_program)
 
root.mainloop()

Thank you for posting. Waiting sounds better than unnecessary checking. I will try to implement this solution.

[philipbergwerf]

I think you could improve this by using a Threading.Condition instance so that instead of a periodic loop, the thread simply sits waiting for user input which is indicated by notifying the Condition.

from tkinter import *
import time
from threading import Condition, Thread
 
root = Tk()
 
text = Text(root)
text.pack()
 
program_is_running = True
 
def render(event=None):
    '''
    This functions represents a really
    heavy render function that can render
    a lot of pages.
    '''
    print('render')

class QuitThread(Exception):
    pass

class ThreadAutoRender(Thread):
    def __init__(self):
        Thread.__init__(self)
        self.needs_to_render = False
        self.condition = Condition()
    def run(self):
        try:
            while True:
                self.wait_and_render()
        except QuitThread:
            pass
        
    def wait_and_render(self):
        with self.condition:
            if not program_is_running:
                raise QuitThread
            while not self.needs_to_render:
                self.condition.wait()
                if not program_is_running:
                    raise QuitThread
            self.needs_to_render = False
        render()

thread_auto_render = ThreadAutoRender()
thread_auto_render.start()

'''
If we close the program program_is_running
will turn False so the render thread is ending
'''
def quit_program(event):
    global program_is_running
    with thread_auto_render.condition:
        program_is_running = False
        thread_auto_render.condition.notify()
    thread_auto_render.join()
    root.destroy()
 
'''
it only needs to render when we type something
in the text box.
'''
def on_keyrelease(event):
    with thread_auto_render.condition:
        thread_auto_render.needs_to_render = True
        thread_auto_render.condition.notify()
 
text.bind('<KeyRelease>', on_keyrelease)
root.bind('<Escape>', quit_program)
 
root.mainloop()

I have a question about this. What happens if there is a python error inside the render() function because I did not fix all bugs? until now it did just retry the render function every time. Does it end the thread?

[philipbergwerf]

I have a follow up question that's connected. The problem I am facing now is that even on a one page score the user sees actually how the computer is drawing. It looks really cool at first but after a few hours of writing it is becoming annoying because it does this drawing every time you press a key. My solution is to first draw a big white rectangle the size of all pages it's going to draw. then I want to draw the whole score below the rectangle so the rectangle covers the unwanted animated drawing and finally remove the rectangle to show the drawing.

I have tried to tag_raise('cover') every time I draw something in the render() function but then I still see every new line or oval for a really short time.

How can I draw a rectangle that stays always on top of the drawing? Is there a way to do this?

[Gribouillis]

What happens if there is a python error inside the render() function because I did not fix all bugs?

A simple solution for this is to call instead a new function render_no_error() that cannot raise an exception. This function would call render()

def render_no_error():
    try:
        render()
    except Exception as e:
        pass  # handle the exception the way you want

The drawback of this technique is to silently suppress exceptions, which makes debugging hard. You will need to leave a trace of the exceptions somewhere, perhaps on stderr or better in a log file or perhaps display it in a tkinter window. You could also catch specific exception types before the Exception type.

it is becoming annoying because it does this drawing every time you press a key

I have been using for a long time a Python program to write mathematical documents. I type in an editor and a subprocess parses what I type and calls other programs to create a Pdf file. I see two windows: the editor and the pdf viewer. The event that triggers the recalculation ot the pdf file is not every keystroke. Instead I recalculate the pdf file every time I type Ctrl-S in the editor to save the source file. This is a very fluid workflow. I suggest that you bind a sequence such as Ctrl-S to the Text or the root widget that would simultaneously save your work and redraw the canvas. It allows you to type as many characters as you want without triggering a redraw.

I'm not a very good tkinter programmer because I don't use it very often but I can imagine solutions to your idea of hiding the canvas when you don't want to see the computer drawing. For example the canvas could be on a page of a Notebook widget and you would simply display another empty page.

[philipbergwerf]

I concluded that it's not possible to draw a new element in tkinter canvas in the background. But I found a solution to the problem of hiding the render and scaling while rendering. The solution is to only show the final render in a second viewport. After rendering the big drawing of a whole score I run these four lines:

# make the new render update fluently
        canvas.move('all', 10000, 0)
        canvas.delete('old')
        canvas.configure(scrollregion=canvas.bbox("all"))
        canvas.addtag_all('old')

So we move the whole rendering 10000 pixels to the right, delete all elements with tag 'old', set the scroll region which is now 10000 pixels to the right, and finally assign the tag 'old' to all elements. Every render it will move and replace the new drawing with the new one using canvas.move(). I found out that moving already drawn elements takes way less computation power so we only see a really short blink between renders which is the same experience as using lilypond.