5 threading methods, none sped up the program

[bobsmith76]

Did you profile the code to find the bottlenecks?

I doubt there would be bottlenecks because all the code does is open a text file and then loop through each line in that text file.

---

Also, I don't understand your use_coroutines() function, since it... doesn't use coroutines lol.

I just adapted the following code from Beazley's Python Cookbook for my own purposes:

# A very simple example of a coroutine/generator scheduler

# Two simple generator functions
def countdown(n):
    while n > 0:
        print("T-minus", n)
        yield
        n -= 1
    print("Blastoff!")

def countup(n):
    x = 0
    while x < n:
        print("Counting up", x)
        yield
        x += 1

from collections import deque

class TaskScheduler:
    def __init__(self):
        self._task_queue = deque()

    def new_task(self, task):
        '''
        Admit a newly started task to the scheduler
        '''
        self._task_queue.append(task)

    def run(self):
        '''
        Run until there are no more tasks
        '''
        while self._task_queue:
            task = self._task_queue.popleft()
            try:
                # Run until the next yield statement
                next(task)
                self._task_queue.append(task)
            except StopIteration:
                # Generator is no longer executing
                pass

# Example use
sched = TaskScheduler()
sched.new_task(countdown(10))
sched.new_task(countdown(5))
sched.new_task(countup(15))
sched.run()

---

It appears that you are reading some of the same files in different threads. You start in different places, but starting with 1, will still read 2, which possibly will conflict with the process starting with 2. Read the directory and split the files up into 5 pieces and try passing each one of the 5 pieces to a different Process to see if this is the problem.

Could you show a simplified example because I don't understand exactly what you're referring to?