Python Forum
Queue.Queue() would not reduce capacity after get()
Thread Rating:
  • 1 Vote(s) - 5 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Queue.Queue() would not reduce capacity after get()
#3
Hi @Windspar,
Thank you for reply and I thought your idea is great.
I did some experiment today. My observation is that the q size is empty, however the system memory is occupied about 84M. Here is some code to reproduce my result.
Ubuntu 16.04, python 2.7.12
First shoot: del
import Queue

q = Queue.Queue()
length = 10000000
buffer_size = 1000
index = 0
while index < length:
  q.put_nowait(1)
  index += 1
key = raw_input('finish insert, press key to pop')
while q.qsize() > buffer_size:
  a = q.get()
  del a
print 'after pop, q size = ', q.qsize()
raw_input('let me del the q')
del q
key = raw_input('finish delete')
Second shoot: clear()
import Queue

q = Queue.Queue()
length = 10000000
buffer_size = 1000
index = 0
while index < length:
  q.put_nowait(1)
  index += 1
key = raw_input('finish insert, press key to pop')
while q.qsize() > buffer_size:
  a = q.get()
  del a
print 'after pop, q size = ', q.qsize()
raw_input('let me del the q')
with q.mutex:
  q.queue.clear()
print 'q size = ', q.qsize()
key = raw_input('finish delete')
Third shoot: Queue()
import Queue

q = Queue.Queue()
length = 10000000
buffer_size = 1000
index = 0
while index < length:
  q.put_nowait(1)
  index += 1
key = raw_input('finish insert, press key to pop')
while q.qsize() > buffer_size:
  a = q.get()
  del a
print 'after pop, q size = ', q.qsize()
raw_input('let me del the q')
q = Queue.Queue()
print 'q size = ', q.qsize()
key = raw_input('finish delete')
Fourth shoot: gc.collect()
import Queue
import gc

q = Queue.Queue()
length = 10000000
buffer_size = 1000
index = 0
while index < length:
  q.put_nowait(1)
  index += 1
key = raw_input('finish insert, press key to pop')
while q.qsize() > buffer_size:
  a = q.get()
  del a
print 'after pop, q size = ', q.qsize()
raw_input('let me del the q')
#del q
#with q.mutex:
#  q.queue.clear()
q = Queue.Queue()
print 'q size = ', q.qsize()
raw_input('let me gc.collect')
gc.collect()
raw_input('how about now?')
These four ways would not release the memory in the queue.Can anyone tell me what I'm doing wrong? Many thanks!



(Dec-29-2017, 04:47 PM)Windspar Wrote: http://effbot.org/pyfaq/why-doesnt-pytho...object.htm

https://stackoverflow.com/questions/1545...-in-python

You could try create a new queue. Swap it out for the old one. Clear old queue then del old queue.
Reply


Messages In This Thread
RE: Queue.Queue() would not reduce capacity after get() - by yuan8421 - Dec-29-2017, 05:53 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  queue for async function python telegram.ext noctious 0 1,724 Jun-11-2023, 02:58 PM
Last Post: noctious
  reduce nested for-loops Phaze90 11 2,072 Mar-16-2023, 06:28 PM
Last Post: ndc85430
  Adding values with reduce() function from the list of tuples kinimod 10 2,888 Jan-24-2023, 08:22 AM
Last Post: perfringo
  Priority Queue with update functionality PythonNewbee 4 2,136 Dec-29-2022, 12:13 PM
Last Post: Yoriz
  Multiprocessing queue catch get timeout Pythocodras 1 2,473 Apr-22-2022, 06:01 PM
Last Post: Pythocodras
  Asyncio: Queue consumer gets out of while loop without break. Where exactly and how? saavedra29 2 2,805 Feb-07-2022, 07:24 PM
Last Post: saavedra29
  Clearing an asyncio queue if it's empty H84Gabor 0 1,941 Jan-28-2022, 02:27 AM
Last Post: H84Gabor
  Error when using Watchdog and Qt to keep open a Queue before and after sending first pyhill00 0 1,677 Oct-28-2021, 09:10 AM
Last Post: pyhill00
  Package cannot be loaded into PBS queue file emersonpl 1 1,913 Sep-09-2021, 08:06 PM
Last Post: emersonpl
  How do I reduce the time to Invoke Macro via Python? JaneTan 1 2,224 Dec-28-2020, 06:46 AM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020