multiprocess hang when certain number is used in the program

[esphi]

Hi, I am a python beginner, I am starting to learn multiprocessing. I have created this program to simply calculate the squares of a big list of numbers.

The program takes a start_number and a end_number, separates the numbers into groups, then uses multiprocessing to calculate the results.

Each process puts the results in a List, and form a dictionary with their sequence number as key, then put into a Queue.

At the end, the program then combine the results as a list and print out the result list.



The program works fine with the start_number = 1, and end_number = any number between 1 to 12_998.

However, the program does not work when end_number = 12,999. It may work with certain end_number and not others passing 12_999. For example it works with 13_000 thru 13_005 but not 13_006 and not 13_007. It works with 13_008 thru 13_012 but not 13_013 ..... and so on.

I have tried running the same program in different computers with different CPU counts, and both windows and linux. The results are the same. I am using python 3.6.9

I have learned later that multiprocess.Pool is easier to use in this scenario. But I am interested to know what was wrong with my multiprocess.Process program, if anyone is kind enough to have a look.

Thank you.

-----------------
My codes:
-----------------

# This program hang when end number = 12_999 , 13_006, 13_007, 13_013.. etc
from multiprocessing import Process, Queue
from multiprocessing import log_to_stderr, get_logger


# divide the range of numbers into groups. so that each group can be processed with different Process. 
def dividegroup(minno, maxno, nof_groups):
    
    total_range = maxno - minno + 1
    remain = total_range % nof_groups
    group_range = total_range // nof_groups

    print(total_range, group_range, remain)

    lof_groups = []
    
    for i in range(nof_groups):
        lof_groups.append( range( minno + (i * group_range), minno + ((i + 1) * group_range )))

    if remain != 0 : lof_groups.append(range(minno + ((i + 1) * group_range), maxno + 1))

    return lof_groups


# square the numbers and put in Queque
def square(seq, numbers, q):

    answers = [x * x for x in numbers]    
    results = {seq: answers} 
    q.put(results, block = False)

# main program
def main():

    log_to_stderr()
    logger = get_logger()
    logger.setLevel(20)

    print('\033c')
    start_number = 1
    end_number = 12_999
    number_of_groups = 8

    list_of_groups = dividegroup(start_number, end_number, number_of_groups)
    print(list(list_of_groups))

    q = Queue(maxsize=0)
    process_seq = 0
    processes = []
 
    for i in list_of_groups:
        process_seq += 1
        process = Process(target = square, args = (process_seq, i, q))
        processes.append(process)

    for process_s in processes:
        process_s.start()

    for process_j in processes:
        process_j.join()

    result_dic = {}
    while not q.empty():
        result_dic.update(q.get())       

    result_list = []
    keylist = list(result_dic.keys())
    keylist.sort()
    for i in keylist:
        result_list += result_dic.get(i)
    print(result_list)


if __name__ == '__main__':
    main()

----------------------------------------------------
The Result when end_number = 12_999 is used.
----------------------------------------------------

Output:
12999 1624 7
[range(1, 1625), range(1625, 3249), range(3249, 4873), range(4873, 6497), range(6497, 8121), range(8121, 9745), range(9745, 11369), range(11369, 12993), range(12993, 13000)]
[INFO/Process-3] child process calling self.run()
[INFO/Process-1] child process calling self.run()
[INFO/Process-3] process shutting down
[INFO/Process-2] child process calling self.run()
[INFO/Process-4] child process calling self.run()
[INFO/Process-3] process exiting with exitcode 0
[INFO/Process-7] child process calling self.run()
[INFO/Process-1] process shutting down
[INFO/Process-1] process exiting with exitcode 0
[INFO/Process-8] child process calling self.run()
[INFO/Process-9] child process calling self.run()
[INFO/Process-6] child process calling self.run()
[INFO/Process-4] process shutting down
[INFO/Process-8] process shutting down
[INFO/Process-4] process exiting with exitcode 0
[INFO/Process-5] child process calling self.run()
[INFO/Process-9] process shutting down
[INFO/Process-7] process shutting down
[INFO/Process-2] process shutting down
[INFO/Process-5] process shutting down
[INFO/Process-6] process shutting down
[INFO/Process-9] process exiting with exitcode 0
[INFO/Process-2] process exiting with exitcode 0
[INFO/Process-8] process exiting with exitcode 0
[INFO/Process-7] process exiting with exitcode 0
[INFO/Process-6] process exiting with exitcode 0