Python Forum
how does GIL impact threads?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how does GIL impact threads?
#1
how does GIL impact the running of threads? i know i can have each running thread start a command in a separate process since i have tested that. i want to run N such processes for a system with N CPUs. when the command process ends, the thread will end. maybe i can have it put a small object on an instance of queue.SimpleQueue and have the main thread waiting for that. but i'm still not sure how GIL can impact this.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
YouTube - Python's Infamous GIL - Larry Hastings
Reply
#3
Global Interpreter Lock
You can share data between threads but not between processes. For each separate process, I think a new Python starts.
"As they say in Mexico 'dosvidaniya'. That makes two vidaniyas."
https://freedns.afraid.org
Reply
#4
> I think a new Python starts.

or does it just replicate a copy of the already running Python? that's what would happen when the system level fork() is called. but it can reload Python and run it from the beginning. normally a fresh Python expects to start a script from the beginning. but is there a way to start a new Python in the middle of a script?

what i have experimented with, so far, is a thread calling subprocess.call(), which runs (and waits for) a process with a specific command. in cases where running some executable in a process is the goal, this seems to be a fine way to go. but cases where continuing to run the same script (maybe in an import module) is a different issue. that replicates everything and the child process would still have threads and the GIL would still be present. maybe it is not safe to fork a process from a multithread situation. doing so in C is not safe, but that is system level threads where there can literally be another thread running in another core when the fork() syscall happens. but, in Python, the GIL is keeping that from being the case.

i need to run the multiprocess case, i think, because i need multiple tasks running botocore, which apparently is not multithread ready (mulithready). i think i need to stay totally away from threads to do processes that keep a copy of Python running (e.g. never call any exec??() syscall).
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020