Hi Community,
I need your help about Queue memory.
1) I choose Queue as my data structure because I have one thread to feed data to the queue and another thread will take the data
2) The two threads designed to run for days
3) I don't want to limit the queue size, the size of queue could be really long, say ~10k which occupy 10GB. This is fine
4) Problem is when I finish shrink the q size by get() to only 20 items which occupy only ~100MB in memory. I print the size and I'm sure there is only 20 items.
5) But in system level, the whole process still occupy ~10GB
I tried to call
And the thread is always running and python would not decrease capacity of the queue.
My question is: Is there anyway to free those memory which the queue doesn't use for now? I can't find any api to do that.
I need your help about Queue memory.
1) I choose Queue as my data structure because I have one thread to feed data to the queue and another thread will take the data
2) The two threads designed to run for days
3) I don't want to limit the queue size, the size of queue could be really long, say ~10k which occupy 10GB. This is fine
4) Problem is when I finish shrink the q size by get() to only 20 items which occupy only ~100MB in memory. I print the size and I'm sure there is only 20 items.
5) But in system level, the whole process still occupy ~10GB
I tried to call
gc.collect()by myself, memory doesn't change. So my wild guess is that those item from get() is destroyed.
And the thread is always running and python would not decrease capacity of the queue.
My question is: Is there anyway to free those memory which the queue doesn't use for now? I can't find any api to do that.