Oct-12-2018, 11:55 AM
(This post was last modified: Oct-12-2018, 11:55 AM by MeenakshiChowdhary.)
I am trying to convert my python code in to parallel computing using multiprocessing.
This is the code
output obtained:
I have tried to check my images if they are corrupted (they work absolutely fine when not used multiprocessing) using the below code:
output in cmd:
(py36) C:\Users\User\Anaconda\envs\py36>python findingcorruptimages.py
This is the code
import numpy as np import os import cv2 import multiprocessing import time file_path=[] path="C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/" for root,dir,files in os.walk(path): for f in files: file_path.append(os.path.join(root,f)) path1="C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Big/" for root1,dir1,files1 in os.walk(path1): for f1 in files1: file_path.append(os.path.join(root1,f1)) print(len(file_path)) def get_image(filepath): print('this is the received path',filepath) img_original= cv2.imread(filepath) # cv2.imshow('original image',img_original) img_array=np.asarray(img_original) blur= cv2.pyrMeanShiftFiltering(img_original,21,49) gray_image= cv2.cvtColor(blur, cv2.COLOR_BGR2GRAY) ret,thresh= cv2.threshold(gray_image,70,255,cv2.THRESH_BINARY) _, contours,hierarchy =cv2.findContours(thresh, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE) countourimage=cv2.drawContours(img_original,contours,-1,0,3) largest_area=0 for i,c in enumerate(contours): contour_areas=cv2.contourArea(c) if(contour_areas>largest_area): largest_area= contour_areas x_rect,y_rect,w_rect,h_rect=cv2.boundingRect(c) cropped=img_original[y_rect:y_rect+h_rect,x_rect:x_rect+w_rect] cv2.imwrite('C:/Users/User/Anaconda/contourimage.jpg',cropped) print('Working inside the loop',filepath) return if __name__ == '__main__': for images in range(3): print('Started running For loop') p = multiprocessing.Process(target=get_image, args=(file_path[images],)) p.start() print('Done'))output in Spyder (Python 3.7.0) console:
Output:129
Started running For loop
Started running For loop
Started running For loop
Done
I tried to run it through the cmd prompt after creating another environment Python 3.6.0output obtained:
Output:(py36) C:\Users\User\Anaconda\envs\py36>python examples.py
129
Started running For loop
Started running For loop
Started running For loop
Done
129
Done
this is the received path C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\blue.jpg
129
Done
129
Done
this is the received path C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\green.jpg
this is the received path C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\red.jpg
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
Corrupt JPEG data: premature end of data segment
Working inside the loop C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\blue.jpg
Working inside the loop C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\red.jpg
Working inside the loop C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small/screen_1\green.jpg
I get this error Error:Corrupt JPEG data: premature end of data segment
for which probable reason could be corrupt images as per Corrupt JPEG dataI have tried to check my images if they are corrupted (they work absolutely fine when not used multiprocessing) using the below code:
import os from subprocess import Popen,PIPE def checkImage(filePath): proc=Popen(['identify','-verbose', filePath],stdout=PIPE, stderr=PIPE) out,err= proc.communicate() exitcode= proc.returncode return exitcode,out,err file_path=[] path="C:/Users/User/Desktop/Dataset_Screens/picture_dataset_Small" for root,dir,files in os.walk(path): for f in files: filepath= os.path.join(root,f) code,output,error = checkImage(filepath) if ((str(code)!='0') or (str(error,"utf-8")!="")): print('ERROR'+filepath) else: print('Good Image'+filepath) file_path.append(filepath)Source
output in cmd:
(py36) C:\Users\User\Anaconda\envs\py36>python findingcorruptimages.py
Error:Traceback (most recent call last):
File "findingcorruptimages.py", line 20, in <module>
code,output,error = checkImage(filepath)
File "findingcorruptimages.py", line 9, in checkImage
proc=Popen(['identify','-verbose', filePath],stdout=PIPE, stderr=PIPE)
File "C:\Users\User\Anaconda\envs\py36\lib\subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "C:\Users\User\Anaconda\envs\py36\lib\subprocess.py", line 997, in _execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
I am not sure where I am going wrong.. Please help!!