Python Forum
Priority Queue with update functionality
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Priority Queue with update functionality
#1
I'm trying to implement a priority queue which allows me to update details of items already in queue.
I found a simple implementation in the Python documentation, which I'm working through to understand (https://docs.python.org/2/library/heapq....tion-notes).
However, there's a portion of this example which the understanding of escapes me; and I'm hoping someone can help me clarify?
The specific example is:
import itertools
import heapq

pq = []                         # list of entries arranged in a heap
entry_finder = {}               # mapping of tasks to entries
REMOVED = '<removed-task>'      # placeholder for a removed task
counter = itertools.count()     # unique sequence count

def add_task(task, priority=0):
    'Add a new task or update the priority of an existing task'
    if task in entry_finder:
        remove_task(task)
    count = next(counter)
    entry = [priority, count, task]
    entry_finder[task] = entry
    heappush(pq, entry)

def remove_task(task):
    'Mark an existing task as REMOVED.  Raise KeyError if not found.'
    entry = entry_finder.pop(task)
    entry[-1] = REMOVED

def pop_task():
    'Remove and return the lowest priority task. Raise KeyError if empty.'
    while pq:
        priority, count, task = heappop(pq)
        if task is not REMOVED:
            del entry_finder[task]
            return task
    raise KeyError('pop from an empty priority queue')

#Testing the above:
add_task('Planning')
add_task('Design',1)
add_task('Building',2)
print(pq)
#output is: [[0, 0, 'Planning'], [1, 1, 'Design'], [2, 2, 'Building']]
add_task('Design',1) #this should update an existing entry
print(pq)
'''
output:
[[0, 0, 'Planning'],
 [1, 1, '<removed-task>'],
 [2, 2, 'Building'],
 [1, 3, 'Design']]
'''
When I work through the example I'm struggling to understand how the second item in the queue gets updated when the second add_task('Design',1) is executed.
The remove_task(task) pops the item in the dictionary to a variable, and the last components of the variable's content is set to REMOVED.
However, that variable only lives in the scope of the remove_task() function, and nowhere does the add_task() or remove_task() functions update the queue when the item is marked for removal.
So I do not understand how this line: [1, 1, '<removed-task>'], comes into existence when printing the queue.
I only see reference to the dictionalry "entry_finder", but nowhere to I see code which updates the item in the queue to "<removed-task>", so how did it happen?

Can anyone explain?

Much appreciated
Reply
#2
The document that you refer to is for python 2 which hasen't been supported since Jan 1, 2020

you can use: https://docs.python.org/3/library/queue.html which has a bulit-in priority queue

Are you using version 2.something?
Reply
#3
(Dec-29-2022, 05:09 AM)PythonNewbee Wrote: I'm trying to implement a priority queue which allows me to update details of items already in queue.
I found a simple implementation in the Python documentation, which I'm working through to understand (https://docs.python.org/2/library/heapq....tion-notes).
However, there's a portion of this example which the understanding of escapes me; and I'm hoping someone can help me clarify?
The specific example is:
import itertools
import heapq

pq = []                         # list of entries arranged in a heap
entry_finder = {}               # mapping of tasks to entries
REMOVED = '<removed-task>'      # placeholder for a removed task
counter = itertools.count()     # unique sequence count

def add_task(task, priority=0):
    'Add a new task or update the priority of an existing task'
    if task in entry_finder:
        remove_task(task)
    count = next(counter)
    entry = [priority, count, task]
    entry_finder[task] = entry
    heappush(pq, entry)

def remove_task(task):
    'Mark an existing task as REMOVED.  Raise KeyError if not found.'
    entry = entry_finder.pop(task)
    entry[-1] = REMOVED

def pop_task():
    'Remove and return the lowest priority task. Raise KeyError if empty.'
    while pq:
        priority, count, task = heappop(pq)
        if task is not REMOVED:
            del entry_finder[task]
            return task
    raise KeyError('pop from an empty priority queue')

#Testing the above:
add_task('Planning')
add_task('Design',1)
add_task('Building',2)
print(pq)
#output is: [[0, 0, 'Planning'], [1, 1, 'Design'], [2, 2, 'Building']]
add_task('Design',1) #this should update an existing entry
print(pq)
'''
output:
[[0, 0, 'Planning'],
 [1, 1, '<removed-task>'],
 [2, 2, 'Building'],
 [1, 3, 'Design']]
'''
When I work through the example I'm struggling to understand how the second item in the queue gets updated when the second add_task('Design',1) is executed.
The remove_task(task) pops the item in the dictionary to a variable, and the last components of the variable's content is set to REMOVED.
However, that variable only lives in the scope of the remove_task() function, and nowhere does the add_task() or remove_task() functions update the queue when the item is marked for removal.
So I do not understand how this line: [1, 1, '<removed-task>'], comes into existence when printing the queue.
I only see reference to the dictionalry "entry_finder", but nowhere to I see code which updates the item in the queue to "<removed-task>", so how did it happen?

Can anyone explain?

Much appreciated

I think I have the answer. Is it that the queue and entry_finder entries are pointers to the original variables I declared, therefore when "remove_task()" pops the entry the new variable "entry" is another pointer to the original. If so, then setting entry[-1] actually changes the original variable and therefore the queue is updated because it contains a pointer to the variable?
Reply
#4
(Dec-29-2022, 11:07 AM)Larz60+ Wrote: The document that you refer to is for python 2 which hasen't been supported since Jan 1, 2020

you can use: https://docs.python.org/3/library/queue.html which has a bulit-in priority queue

Are you using version 2.something?


No, using version 3.8.
I will check out the document you mentioned, thank you
Reply
#5
In the function add_task a list is created entry = [priority, count, task]
A pointer to the same list is added to entry_finder and pqin the lines
entry_finder[task] = entry
heapq.heappush(pq, entry)
In the function remove_task in the lines
entry = entry_finder.pop(task)
entry[-1] = REMOVED
It pops off the pointer to a list from entry_finder and then alters the last item in that list because that same list is also in pq it will also show the alteration.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Randomizer script to provide options based on user and priority cubangt 10 2,471 Mar-11-2022, 07:22 PM
Last Post: cubangt
  Putting code into a function breaks its functionality, though the code is identical! PCesarano 1 2,003 Apr-05-2021, 05:40 PM
Last Post: deanhystad
Question Python + Google Sheet | Best way to update specific cells in a single Update()? Vokofe 1 2,700 Dec-16-2020, 05:26 AM
Last Post: Vokofe
  task queue Valon1981 8 3,614 Jul-07-2020, 07:41 AM
Last Post: freeman
  Queue in Pygame constantin01 1 3,697 Jan-07-2020, 04:02 PM
Last Post: metulburr
  Validating the functionality of the application rpalakodety 1 1,779 Dec-30-2019, 07:58 PM
Last Post: ndc85430
  Queue maxsize mr_byte31 2 4,567 Sep-03-2019, 07:02 PM
Last Post: mr_byte31
  I'm having trouble with an OOP version of Pickle functionality CodeWolf 2 2,376 Dec-19-2018, 05:41 PM
Last Post: CodeWolf
  Queue.Queue() would not reduce capacity after get() yuan8421 9 11,107 Jan-02-2018, 09:38 PM
Last Post: Windspar
  Threading and Queue nexusfactor 5 4,292 Oct-16-2017, 04:14 PM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020