Python Forum
How to run existing python script parallel using multiprocessing
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How to run existing python script parallel using multiprocessing
#4
Hi,

I had a python script that searches for an item in a list.
Now my requirement is to start a separate thread for each item search in the for loop.
How can I achieve this?
Please help
import glob
import os
import csv
patterns = ['ACM','ACX','AW','BC']
for pattern in patterns:
    base_path = '/ai2/data/dev/admin/inf/*{}*'
    search_path =  base_path.format(pattern)
    for f in glob.glob(search_path):
        print("-----------------------")
        print ("The directory path is:")
        print f
        print("List of files in the directory are:")
        os.chdir('/ai2/data/dev/admin/inf/')
        os.chdir(f)
        cwd = os.getcwd()   
        for subdir, dirs, files in os.walk(cwd, topdown=True):
            for file23 in glob.glob('*.err'):
                print file23
In the above script for every "pattern" value, I need to start a separate thread

Output:
[root@edgenod]# python p123.py ----------------------- The directory path is: /ai2/data/dev/admin/inf/inf_ACM_pvt List of files in the directory are: dsplit.err ----------------------- The directory path is: /ai2/data/dev/admin/inf/inf_ACX_pvt List of files in the directory are: accountshighfocus.err acm_access_log.err ----------------------- The directory path is: /ai2/data/dev/admin/inf/inf_AW_pvt List of files in the directory are: aware.err ----------------------- The directory path is: /ai2/data/dev/admin/inf/inf_BC_pvt List of files in the directory are: run_ingest_BC_daily_1249.err
Reply


Messages In This Thread
RE: How to run existing python script parallel using multiprocessing - by lravikumarvsp - May-24-2018, 05:23 AM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Multiprocessing on python sawtooth500 12 546 Apr-02-2024, 06:03 PM
Last Post: sawtooth500
  Python - Merge existing cells of Excel file created with xlsxwriter manonB 0 3,723 Mar-10-2021, 02:17 PM
Last Post: manonB
  Python Parallel Programing wissam1974 6 8,775 Feb-25-2019, 08:48 PM
Last Post: wissam1974
  Updating the Pandas dataframe to existing excel workbook in existing worksheet. sanmaya 1 9,766 Jul-01-2018, 06:23 PM
Last Post: volcano63

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020