Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Why thread and queue only process the last row
#1
CSv
date,ref_num,desc,debit,credit,payee_id,code,name

    2019-01-31,L00001,john,30.00,,150,500,johnkino
    2019-01-31,L00001,john,,30.00,150,600,johnkino
    2019-01-30,L00002,john,30.00,,150,500,johnkino
    2019-01-30,L00002,john,,30.00,150,600,johnkino
import csv
from itertools import groupby
import threading
from queue import Queue

def postRequest():
    print('thread starting')
    while True:
        item_to_process = q.get()
        print(item_to_process)
        q.task_done()

q = Queue()

with open("test.csv", "r") as csv_ledger:
    r = csv.DictReader(csv_ledger)
    data = [ dict(d) for d in r ]
    groups = {}

    for k, g in groupby(data, lambda r: (r[ 'ref_num' ])):
        items = [ ]
        for i in g:
            chart_of_account = {k: v for k, v in i.items() if k in [ 'name', 'code' ]}
            item = {k: v for k, v in i.items() if k in [ 'debit', 'credit', 'desc', 'payee_id' ]}
            item.update({'chart_of_account': chart_of_account})
            items.append(item)
        groups.update({
            "date": i[ 'date' ],
            "desc": i[ 'desc' ],
            "ref_num": k,
            "items": items
        })
        print(groups) <--- correct structure
        q.put(groups)


for i in range(1):
    t = threading.Thread(target=postRequest)
    t.start()

print('Main thread waiting')
q.join()
print('Done.')
Result
2019-01-30,L00002,john,,30.00,150,600,johnkino
2019-01-30,L00002,john,,30.00,150,600,johnkino
2019-01-30,L00002,john,,30.00,150,600,johnkino
2019-01-30,L00002,john,,30.00,150,600,johnkino
The data of groups is the complete and correct data structure which is loop every row but the result always loops for the last row only after passing to queue for processing.
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  How to sharing object between multiple process from main process using Pipe Subrata 1 369 Sep-03-2019, 09:49 PM
Last Post: woooee
  multiprocessing.Queue.join_thread() says "thread" Skaperen 0 170 Jul-29-2019, 12:45 AM
Last Post: Skaperen
  Queue.Queue() would not reduce capacity after get() yuan8421 9 2,636 Jan-02-2018, 09:38 PM
Last Post: Windspar
  Good way to have a worker queue accessible by multiple process? cheater 2 778 Dec-21-2017, 09:30 PM
Last Post: wavic

Forum Jump:


Users browsing this thread: 1 Guest(s)