Nov-12-2019, 05:43 AM
I've developed an image processing script that does gamma correction (on cpu), sobel(on nvidia gpu).
But, strangely the program is crashing frequently where the frequency lies anywhere between 4 hours to 14 hours. I've no clue how to debug this error as it gives only one error message in stack trace "Fatal Python Error".
Packages/Libraries used:
numba.cuda, numpy, sklearn.cluster, cv2, multiprocessing.Pool
Any idea how to debug/resolve this python crash error.
Please see the below stack trace of the error.
But, strangely the program is crashing frequently where the frequency lies anywhere between 4 hours to 14 hours. I've no clue how to debug this error as it gives only one error message in stack trace "Fatal Python Error".
Packages/Libraries used:
numba.cuda, numpy, sklearn.cluster, cv2, multiprocessing.Pool
Any idea how to debug/resolve this python crash error.
Please see the below stack trace of the error.
======= Backtrace: =========
INFO:cuda_img_process:53760
/lib64/libc.so.6(+0x816b9)[0x7f7d483f96b9]
/opt/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/../../../../../libzmq.so.5(+0x328ee)[0x7f7d2808a8ee]
/opt/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/../../../../../libzmq.so.5(+0x6631a)[0x7f7d280be31a]
/opt/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/../../../../../libzmq.so.5(+0x5407a)[0x7f7d280ac07a]
/opt/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/../../../../../libzmq.so.5(+0x27561)[0x7f7d2807f561]
/opt/anaconda3/lib/python3.6/site-packages/zmq/backend/cython/../../../../../libzmq.so.5(+0x5df86)[0x7f7d280b5f86]
/lib64/libpthread.so.0(+0x7ea5[2019-11-11 20:39:43,323] -- INFO - cuda_roi_image_process_normalize_writeimage_api2.py -- receive_image_for_image_process -- Line no - 370 -- CUDA Device : 2
/lib64/libc.so.6(clone+0x6d)[0x7f7d484768cd]
======= Memory map: ========
INFO:cuda_img_process:CUDA Device : 2
[2019-11-11 20:39:43,323] -- INFO - cuda_roi_image_process_normalize_writeimage_api2.py -- receive_image_for_image_process -- Line no - 382 -- Image Recieved for Image Processing : 53761 : 111129_1_261.bmp
INFO:cuda_img_process:Image Recieved for Image Processing : 53761 : 111129_1_261.bmp
200000000-200200000 rw-s 00000000 00:05 92844 /dev/nvidiactl
200200000-200600000 ---p 00000000 00:00 0
200600000-200800000 rw-s 00000000 00:05 92844 /dev/nvidiactl
200800000-200a00000 rw-s 00000000 00:05 92847 /dev/nvidia2
200a00000-205200000 rw-s 00000000 00:05 92844 /dev/nvidiactl
205200000-205400000 rw-s 00000000 00:05 92847 /dev/nvidia2
205400000-206400000 ---p 00000000 00:00 0
206400000-206600000 rw-s 00000000 00:05 92844 /dev/nvidiactl
206600000-206800000 rw-s 00000000 00:05 92844 /dev/nvidiactl
206800000-206a00000 rw-s 206800000 00:05 99322 /dev/nvidia-uvm
206a00000-206c00000 ---p 00000000 00:00 0
206c00000-206e00000 rw-s 00000000 00:05 92844 /dev/nvidiactl
206e00000-207000000 rw-s 00000000 00:04 142114037 /dev/zero (deleted)
207000000-600200000 ---p 00000000 00:00 0
10000000000-10004000000 ---p 00000000 00:00 0
55858734d000-5585873a4000 r--p 00000000 fd:00 843121090 /opt/anaconda3/bin/python3.6
5585873a4000-55858756b000 r-xp 00057000 fd:00 843121090 /opt/anaconda3/bin/python3.6
55858756b000-558587608000 r--p 0021e000 fd:00 843121090 /opt/anaconda3/bin/python3.6
558587609000-55858760c000 r--p 002bb000 fd:00 843121090 /opt/anaconda3/bin/python3.6
55858760c000-55858766f000 rw-p 002be000 fd:00 843121090 /opt/anaconda3/bin/python3.6
55858766f000-5585876a0000 rw-p 00000000 00:00 0
5585886f6000-55858ab8b000 rw-p 00000000 00:00 0 [heap]
55858ab8b000-55858abab000 rw-p 00000000 00:00 0 [heap]
55858abab000-55858b5da000 rw-p 00000000 00:00 0 [heap]
55858b5da000-55858b5fa000 rw-p 00000000 00:00 0 [heap]
55858b5fa000-55858c05f000 rw-p 00000000 00:00 0 [heap]
55858c05f000-55858c07b000 rw-p 00000000 00:00 0 [heap]
55858c07b000-55858cb32000 rw-p 00000000 00:00 0 [heap]
55858cb32000-55858d5b1000 rw-p 00000000 00:00 0 [heap]
55858d5b1000-55858de72000 rw-p 00000000 00:00 0 [heap]
55858de72000-55858e00b000 rw-p 00000000 00:00 0 [heap]
55858e00b000-55858e039000 rw-p 00000000 00:00 0 [heap]
55858e039000-55858ea94000 rw-p 00000000 00:00 0 [heap]
55858ea94000-55858ea98000 rw-p 00000000 00:00 0 [heap]
55858ea98000-55858ffcd000 rw-p 00000000 00:00 0 [heap]
55858ffcd000-55858ffdf000 rw-p 00000000 00:00 0 [heap]
55858ffdf000-558590b16000 rw-p 00000000 00:00 0 [heap]
558590b16000-558592466000 rw-p 00000000 00:00 0 [heap]
558592466000-55859273b000 rw-p 00000000 00:00 0 [heap]
55859273b000-558593969000 rw-p 00000000 00:00 0 [heap]
558593969000-5585943c9000 rw-p 00000000 00:00 0 [heap]
5585943c9000-558594e70000 rw-p 00000000 00:00 0 [heap]
558594e70000-55859529c000 rw-p 00000000 00:00 0 [heap]
55859529c000-55859635b000 rw-p 00000000 00:00 0 [heap]
55859635b000-558596d9d000 rw-p 00000000 00:00 0 [heap]
558596d9d000-558598287000 rw-p 00000000 00:00 0 [heap]
558598287000-558599132000 rw-p 00000000 00:00 0 [heap]
558599132000-55859aedd000 rw-p 00000000 00:00 0 [heap]
55859aedd000-55859aefd000 rw-p 00000000 00:00 0 [heap]
7f77b8000000-7f77b8001000 rw-p 00000000 00:00 0
7f77b8001000-7f77b94fb000 rw-p 00000000 00:00 0
7f77b94fb000-7f77ba3cd000 rw-p 00000000 00:00 0
7f77ba3cd000-7f77bb29f000 rw-p 00000000 00:00 0
[2019-11-11 20:39:43,324] -- INFO - cuda_roi_image_process_normalize_writeimage_api2.py -- cuda_process_roi_extract -- Line no - 766 -- Splitted image received for processing : 111129_1_261_0_0_0.bmp
7f77bb29f000-7f77bb49c000 rw-p 00000000 00:00 0
7f77bb49c000-7f77bbf39000 rw-p 00000000 00:00 0
7f77bbf39000-7f77bbf93000 rw-p 00000000 00:00 0
7f77bbf93000-7f77bc000000 ---p 00000000 00:00 0
7f77c8000000-7f77c8021000 rw-p 00000000 00:00 0
7f77c8021000-7f77cc000000 ---p 00000000 00:00 0
7f77cc000000-7f77cc021000 rw-p 00000000 00:00 0
7f77cc021000-7f77d0000000 ---p 00000000 00:00 0
7f77d0000000-7f77d0021000 rw-p 00000000 00:00 0
7f77d0021000-7f77d4000000 ---p 00000000 00:00 0
7f77d4000000-7f77d4021000 rw-p 00000000 00:00 0
7f77d4021000-7f77d8000000 ---p 00000000 00:00 0
7f77d8000000-7f77d8021000 rw-p 00000000 00:00 0
7f77d8021000-7f77dc000000 ---p 00000000 00:00 0
7f77dc000000-7f77dc021000 rw-p 00000000 00:00 0
7f77dc021000-7f77e0000000 ---p 00000000 00:00 0
7f77e1ff0000-7f77e1ff1000 ---p 00000000 00:00 0
7f77e1ff1000-7f77e27f4000 rwxp 00000000 00:00 0 [stack:398300]
7f77e27f4000-7f77e27f5000 ---p 00000000 00:00 0
7f77e27f5000-7f77e2ff8000 rwxp 00000000 00:00 0 INFO:cuda_img_process:Splitted image received for processing : 111129_1_261_0_0_0.bmp
[stack:398299]
7f77e2ff8000-7f77e2ff9000 ---p 00000000 00:00 0
7f77e2ff9000-7f77e37fc000 rwxp 00000000 00:00 0 [stack:398298]
7f77e37fc000-7f77e37fd000 ---p 00000000 00:00 0
7f77e37fd000-7f77e4000000 rwxp 00000000 00:00 0 [stack:398297]
7f77e4000000-7f77e4021000 rw-p 00000000 00:00 0
7f77e4021000-7f77e8000000 ---p 00000000 00:00 0
7f77e8000000-7f77e8021000 rw-p 00000000 00:00 0
7f77e8021000-7f77ec000000 ---p 00000000 00:00 0
7f77ec000000-7f77ec021000 rw-p 00000000 00:00 0
7f77ec021000-7f77f0000000 ---p 00000000 00:00 0
7f77f0000000-7f77f0021000 rw-p 00000000 00:00 0
7f77f0021000-7f77f4000000 ---p 00000000 00:00 0
7f77f4000000-7f77f4021000 rw-p 00000000 00:00 0
7f77f4021000-7f77f8000000 ---p 00000000 00:00 0
7f77f87e4000-7f77f87e5000 ---p 00000000 00:00 0
7f77f87e5000-7f77f8fe8000 rwxp 00000000 00:00 0 [stack:398296]
7f77f8fe8000-7f77f8fe9000 ---p 00000000 00:00 0
7f77f8fe9000-7f77f97ec000 rwxp 00000000 00:00 0 [stack:398295]
7f77f97ec000-7f77f97ed000 ---p 00000000 00:00 0
7f77f97ed000-7f77f9ff0000 rwxp 00000000 00:00 0 [stack:398294]
7f77f9ff0000-7f77f9ff1000 ---p 00000000 00:00 0
7f77f9ff1000-7f77fa7f4000 rwxp 00000000 00:00 0 [stack:398290]
7f77fa7f4000-7f77fa7f5000 ---p 00000000 00:00 0
7f77fa7f5000-7f77faff8000 rwxp 00000000 00:00 0 [stack:398287]
7f77faff8000-7f77faff9000 ---p 00000000 00:00 0
7f77faff9000-7f77fb7fc000 rwxp 00000000 00:00 0 [stack:398284]
7f77fb7fc000-7f77fb7fd000 ---p 00000000 00:00 0
7f77fb7fd000-7f77fc000000 rwxp 00000000 00:00 0 [stack:398281]
7f77fc000000-7f77fc021000 rw-p 00000000 00:00 0
7f77fc021000-7f7800000000 ---p 00000000 00:00 0
7f7800000000-7f7800021000 rw-p 00000000 00:00 0
7f7800021000-7f7804000000 ---p 00000000 00:00 0
7f7804000000-7f7804021000 rw-p 00000000 00:00 0
7f7804021000-7f7808000000 ---p 00000000 00:00 0
7f7808000000-7f7808021000 rw-p 00000000 00:00 0
7f7808021000-7f780c000000 ---p 00000000 00:00 0
7f780c000000-7f780c021000 rw-p 00000000 00:00 0
7f780c021000-7f7810000000 ---p 00000000 00:00 0
7f7810000000-7f7810021000 rw-p 00000000 00:00 0
7f7810021000-7f7814000000 ---p 00000000 00:00 0
7f7814000000-7f7814021000 rw-p 00000000 00:00 0
7f7814021000-7f7818000000 ---p 00000000 00:00 0
7f7818000000-7f7818021000 rw-p 00000000 00:00 0
7f7818021000-7f781c000000 ---p 00000000 00:00 0
7f781c7e4000-7f781c7e5000 ---p 00000000 00:00 0
7f781c7e5000-7f781cfe8000 rwxp 00000000 00:00 0 [stack:398278]
7f781cfe8000-7f781cfe9000 ---p 00000000 00:00 0
7f781cfe9000-7f781d7ec000 rwxp 00000000 00:00 0 [stack:398275]
7f781d7ec000-7f781d7ed000 ---p 00000000 00:00 0
7f781d7ed000-7f781dff0000 rwxp 00000000 00:00 0 [stack:398273]
7f781dff0000-7f781dff1000 ---p 00000000 00:00 0
7f781dff1000-7f781e7f4000 rwxp 00000000 00:00 0 [stack:398269]
7f781e7f4000-7f781e7f5000 ---p 00000000 00:00 0
7f781e7f5000-7f781eff8000 rwxp 00000000 00:00 0 [stack:398265]
7f781eff8000-7f781eff9000 ---p 00000000 00:00 0
7f781eff9000-7f781f7fc000 rwxp 00000000 00:00 0 [stack:398260]
7f781f7fc000-7f781f7fd000 ---p 00000000 00:00 0
7f781f7fd000-7f7820000000 rwxp 00000000 00:00 0 [stack:398257]
7f7820000000-7f7820021000 rw-p 00000000 00:00 0
7f7820021000-7f7824000000 ---p 00000000 00:00 0
7f7824000000-7f7824021000 rw-p 00000000 00:00 0
7f7824021000-7f7828000000 ---p 00000000 00:00 0
7f7828000000-7f7828021000 rw-p 00000000 00:00 0
7f7828021000-7f782c000000 ---p 00000000 00:00 0
7f782c000000-7f782c021000 rw-p 00000000 00:00 0
7f782c021000-7f7830000000 ---p 00000000 00:00 0
7f7830000000-7f7830021000 rw-p 00000000 00:00 0
7f7830021000-7f7834000000 ---p 00000000 00:00 0
7f7834000000-7f7834021000 rw-p 00000000 00:00 0
7f7834021000-7f7838000000 ---p 00000000 00:00 0
7f7838000000-7f7838021000 rw-p 00000000 00:00 0
7f7838021000-7f783c000000 ---p 00000000 00:00 0
7f783c000000-7f783c021000 rw-p 00000000 00:00 0
7f783c021000-7f7840000000 ---p 00000000 00:00 0
7f7840000000-7f7840021000 rw-p 00000000 00:00 0
7f7840021000-7f7844000000 ---p 00000000 00:00 0
7f7844000000-7f7844021000 rw-p 00000000 00:00 0
7f7844021000-7f7848000000 ---p 00000000 00:00 0
7f7848000000-7f7848021000 rw-p 00000000 00:00 0
7f7848021000-7f784c000000 ---p 00000000 00:00 0
7f784c000000-7f784c021000 rw-p 00000000 00:00 0
7f784c021000-7f7850000000 ---p 00000000 00:00 0
7f7850000000-7f7850021000 rw-p 00000000 00:00 0
7f7850021000-7f7854000000 ---p 00000000 00:00 0
7f7854000000-7f7854021000 rw-p 00000000 00:00 0
7f7854021000-7f7858000000 ---p 00000000 00:00 0
7f7858000000-7f7858021000 rw-p 00000000 00:00 0
7f7858021000-7f785c000000 ---p 00000000 00:00 0
7f785c000000-7f785c021000 rw-p 00000000 00:00 0
7f785c021000-7f7860000000 ---p 00000000 00:00 0
7f7860000000-7f7860021000 rw-p 00000000 00:00 0
7f7860021000-7f7864000000 ---p 00000000 00:00 0
7f7864000000-7f7864021000 rw-p 00000000 00:00 0
7f7864021000-7f7868000000 ---p 00000000 00:00 0
7f7868000000-7f7868021000 rw-p 00000000 00:00 0
7f7868021000-7f786c000000 ---p 00000000 00:00 0
7f786c000000-7f786c021000 rw-p 00000000 00:00 0
7f786c021000-7f7870000000 ---p 00000000 00:00 0
7f7870000000-7f7870021000 rw-p 00000000 00:00 0
7f7870021000-7f7874000000 ---p 00000000 00:00 0
7f7874000000-7f7874021000 rw-p 00000000 00:00 0
7f7874021000-7f7878000000 ---p 00000000 00:00 0
7f7878000000-7f7878021000 rw-p 00000000 00:00 0
7f7878021000-7f787c000000 ---p 00000000 00:00 0
7f787c000000-7f787c021000 rw-p 00000000 00:00 0
7f787c021000-7f7880000000 ---p 00000000 00:00 0
7f7880000000-7f7880021000 rw-p 00000000 00:00 0
7f7880021000-7f7884000000 ---p 00000000 00:00 0
7f7884000000-7f7884021000 rw-p 00000000 00:00 0
7f7884021000-7f7888000000 ---p 00000000 00:00 0
7f7888000000-7f7888021000 rw-p 00000000 00:00 0
7f7888021000-7f788c000000 ---p 00000000 00:00 0
7f788c000000-7f788c021000 rw-p 00000000 00:00 0
7f788c021000-7f7890000000 ---p 00000000 00:00 0
7f7890000000-7f7890021000 rw-p 00000000 00:00 0
7f7890021000-7f7894000000 ---p 00000000 00:00 0
7f7894000000-7f7894021000 rw-p 00000000 00:00 0
7f7894021000-7f7898000000 ---p 00000000 00:00 0
7f7898000000-7f7898021000 rw-p 00000000 00:00 0
7f7898021000-7f789c000000 ---p 00000000 00:00 0
7f789c000000-7f789c021000 rw-p 00000000 00:00 0
7f789c021000-7f78a0000000 ---p 00000000 00:00 0
7f78a0000000-7f78a0021000 rw-p 00000000 00:00 0
7f78a0021000-7f78a4000000 ---p 00000000 00:00 0
7f78a4000000-7f78a4021000 rw-p 00000000 00:00 0
7f78a4021000-7f78a8000000 ---p 00000000 00:00 0
7f78a8000000-7f78a8021000 rw-p 00000000 00:00 0
7f78a8021000-7f78ac000000 ---p 00000000 00:00 0
7f78ac000000-7f78ac021000 rw-p 00000000 00:00 0
7f78ac021000-7f78b0000000 ---p 00000000 00:00 0
7f78b0000000-7f78b0021000 rw-p 00000000 00:00 0
7f78b0021000-7f78b4000000 ---p 00000000 00:00 0
7f78b4000000-7f78b4021000 rw-p 00000000 00:00 0
7f78b4021000-7f78b8000000 ---p 00000000 00:00 0
7f78b8000000-7f78b8021000 rw-p 00000000 00:00 0
7f78b8021000-7f78bc000000 ---p 00000000 00:00 0
7f78bc000000-7f78bc021000 rw-p 00000000 00:00 0
7f78bc021000-7f78c0000000 ---p 00000000 00:00 0
7f78c0000000-7f78c0021000 rw-p 00000000 00:00 0
7f78c0021000-7f78c4000000 ---p 00000000 00:00 0
7f78c4000000-7f78c4021000 rw-p 00000000 00:00 0
7f78c4021000-7f78c8000000 ---p 00000000 00:00 0
7f78c8000000-7f78c8021000 rw-p 00000000 00:00 0
7f78c8021000-7f78cc000000 ---p 00000000 00:00 0
7f78cc000000-7f78cc021000 rw-p 00000000 00:00 0
7f78cc021000-7f78d0000000 ---p 00000000 00:00 0
7f78d0000000-7f78d0021000 rw-p 00000000 00:00 0
7f78d0021000-7f78d4000000 ---p 00000000 00:00 0
7f78d4000000-7f78d4021000 rw-p 00000000 00:00 0
7f78d4021000-7f78d8000000 ---p 00000000 00:00 0
7f78d8000000-7f78d8021000 rw-p 00000000 00:00 0
7f78d8021000-7f78dc000000 ---p 00000000 00:00 0
7f78dc000000-7f78dc021000 rw-p 00000000 00:00 0
7f78dc021000-7f78e0000000 ---p 00000000 00:00 0
7f78e0000000-7f78e0021000 rw-p 00000000 00:00 0
7f78e0021000-7f78e4000000 ---p 00000000 00:00 0
7f78e4000000-7f78e4021000 rw-p 00000000 00:00 0
7f78e4021000-7f78e8000000 ---p 00000000 00:00 0
7f78e8000000-7f78e8021000 rw-p 00000000 00:00 0
7f78e8021000-7f78ec000000 ---p 00000000 00:00 0
7f78ec7da000-7f78ec7db000 ---p 00000000 00:00 0
7f78ec7db000-7f78ecfde000 rwxp 00000000 00:00 0 [stack:398250]
7f78ecfde000-7f78ecfdf000 ---p 00000000 00:00 0
7f78ecfdf000-7f78ed7e2000 rwxp 00000000 00:00 0 [stack:398244]
7f78ed7e2000-7f78ed7e3000 ---p 00000000 00:00 0
7f78ed7e3000-7f78edfe6000 rwxp 00000000 00:00 0 [stack:398243]
7f78edfe6000-7f78edfe7000 ---p 00000000 00:00 0
7f78edfe7000-7f78ee7ea000 rwxp 00000000 00:00 0 [stack:398242]
7f78ee7ea000-7f78ee7eb000 ---p 00000000 00:00 0
7f78ee7eb000-7f78eefee000 rwxp 00000000 00:00 0 [stack:398241]
7f78eefee000-7f78eefef000 ---p 00000000 00:00 0
7f78eefef000-7f78ef7ef000 rwxp 00000000 00:00 0 [stack:397864]
7f78ef7ef000-7f78ef7f0000 ---p 00000000 00:00 0
7f78ef7f0000-7f78efff0000 rwxp 00000000 00:00 0 [stack:397863]
7f78efff0000-7f78efff1000 ---p 00000000 00:00 0
7f78efff1000-7f78f07f1000 rwxp 00000000 00:00 0 [stack:397862]
7f78f07f1000-7f78f07f2000 ---p 00000000 00:00 0
7f78f07f2000-7f78f0ff2000 rwxp 00000000 00:00 0 [stack:397861]
7f78f0ff2000-7f78f0ff3000 ---p 00000000 00:00 0
7f78f0ff3000-7f78f17f3000 rwxp 00000000 00:00 0 [stack:397860]
7f78f17f3000-7f78f17f4000 ---p 00000000 00:00 0
7f78f17f4000-7f78f1ff4000 rwxp 00000000 00:00 0 [stack:397859]
7f78f1ff4000-7f78f1ff5000 ---p 00000000 00:00 0
7f78f1ff5000-7f78f27f5000 rwxp 00000000 00:00 0 [stack:397858]
7f78f27f5000-7f78f27f6000 ---p 00000000 00:00 0
7f78f27f6000-7f78f2ff6000 rwxp 00000000 00:00 0 [stack:397857]
7f78f2ff6000-7f78f2ff7000 ---p 00000000 00:00 0
7f78f2ff7000-7f78f37f7000 rwxp 00000000 00:00 0 [stack:397856]
7f78f37f7000-7f78f37f8000 ---p 00000000 00:00 0
7f78f37f8000-7f78f3ff8000 rwxp 00000000 00:00 0 [stack:397855]
7f78f3ff8000-7f78f3ff9000 ---p 00000000 00:00 0
7f78f3ff9000-7f78f47f9000 rwxp 00000000 00:00 0 [stack:397854]
7f78f47f9000-7f78f47fa000 ---p 00000000 00:00 0
7f78f47fa000-7f78f4ffa000 rwxp 00000000 00:00 0 [stack:397853]
7f78f4ffa000-7f78f4ffb000 ---p 00000000 00:00 0
7f78f4ffb000-7f78f57fb000 rwxp 00000000 00:00 0 [stack:397852]
7f78f57fb000-7f78f57fc000 ---p 00000000 00:00 0
7f78f57fc000-7f78f5ffc000 rwxp 00000000 00:00 0 [stack:397851]
7f78f5ffc000-7f78f5ffd000 ---p 00000000 00:00 0
7f78f5ffd000-7f78f67fd000 rwxp 00000000 00:00 0 [stack:397850]
7f78f67fd000-7f78f67fe000 ---p 00000000 00:00 0
7f78f67fe000-7f78f6ffe000 rwxp 00000000 00:00 0 [stack:397849]
7f78f6ffe000-7f78f6fff000 ---p 00000000 00:00 0
7f78f6fff000-7f78f77ff000 rwxp 00000000 00:00 0 [stack:397848]
7f78f77ff000-7f78f7800000 ---p 00000000 00:00 0
7f78f7800000-7f78f8000000 rwxp 00000000 00:00 0 [stack:397847]
7f78f8000000-7f78f8021000 rw-p 00000000 00:00 0
7f78f8021000-7f78fc000000 ---p 00000000 00:00 0
7f78fc000000-7f78fc021000 rw-p 00000000 00:00 0
7f78fc021000-7f7900000000 ---p 00000000 00:00 0
7f7900000000-7f7900021000 rw-p 00000000 00:00 0
7f7900021000-7f7904000000 ---p 00000000 00:00 0
7f7904000000-7f7904021000 rw-p 00000000 00:00 0
7f7904021000-7f7908000000 ---p 00000000 00:00 0
7f7908000000-7f7908021000 rw-p 00000000 00:00 0
7f7908021000-7f790c000000 ---p 00000000 00:00 0
7f790c000000-7f790c001000 rw-p 00000000 00:00 0
7f790c001000-7f790ced3000 rw-p 00000000 00:00 0
7f790ced3000-7f790cede000 rw-p 00000000 00:00 0
7f790cede000-7f790ddb1000 rw-p 00000000 00:00 0
7f790ddb1000-7f790ec83000 rw-p 00000000 00:00 0
7f790ec83000-7f790fcb6000 rw-p 00000000 00:00 0
7f790fcb6000-7f790ff76000 rw-p 00000000 00:00 0
7f790ff76000-7f7910000000 ---p 00000000 00:00 0
7f7910000000-7f7910021000 rw-p 00000000 00:00 0
7f7910021000-7f7914000000 ---p 00000000 00:00 0
7f7914000000-7f7914021000 rw-p 00000000 00:00 0
7f7914021000-7f7918000000 ---p 00000000 00:00 0
7f7918000000-7f7918021000 rw-p 00000000 00:00 0
7f7918021000-7f791c000000 ---p 00000000 00:00 0
7f791c000000-7f791c021000 rw-p 00000000 00:00 0
7f791c021000-7f7920000000 ---p 00000000 00:00 0
7f7920000000-7f7920021000 rw-p 00000000 00:00 0
7f7920021000-7f7924000000 ---p 00000000 00:00 0
7f7924000000-7f7924021000 rw-p 00000000 00:00 0
7f7924021000-7f7928000000 ---p 00000000 00:00 0
7f7928000000-7f7928021000 rw-p 00000000 00:00 0
7f7928021000-7f792c000000 ---p 00000000 00:00 0
7f792c000000-7f792c021000 rw-p 00000000 00:00 0
7f792c021000-7f7930000000 ---p 00000000 00:00 0
7f7930000000-7f7930021000 rw-p 00000000 00:00 0
7f7930021000-7f7934000000 ---p 00000000 00:00 0
7f7934000000-7f7934021000 rw-p 00000000 00:00 0
7f7934021000-7f7938000000 ---p 00000000 00:00 0
7f7938000000-7f7938021000 rw-p 00000000 00:00 0
7f7938021000-7f793c000000 ---p 00000000 00:00 0
7f793c000000-7f793c021000 rw-p 00000000 00:00 0
7f793c021000-7f7940000000 ---p 00000000 00:00 0
7f7940000000-7f7940021000 rw-p 00000000 00:00 0
7f7940021000-7f7944000000 ---p 00000000 00:00 0
7f7944000000-7f7944021000 rw-p 00000000 00:00 0
7f7944021000-7f7948000000 ---p 00000000 00:00 0
7f7948000000-7f7948021000 rw-p 00000000 00:00 0
7f7948021000-7f794c000000 ---p 00000000 00:00 0
7f794c000000-7f794c021000 rw-p 00000000 00:00 0
7f794c021000-7f7950000000 ---p 00000000 00:00 0
7f7950000000-7f7950021000 rw-p 00000000 00:00 0
7f7950021000-7f7954000000 ---p 00000000 00:00 0
7f7954000000-7f7954021000 rw-p 00000000 00:00 0
7f7954021000-7f7958000000 ---p 00000000 00:00 0
7f7958000000-7f7958021000 rw-p 00000000 00:00 0
7f7958021000-7f795c000000 ---p 00000000 00:00 0
7f795c7f9000-7f795c7fa000 ---p 00000000 00:00 0
7f795c7fa000-7f795cffa000 rwxp 00000000 00:00 0 [stack:397846]
7f795cffa000-7f795cffb000 ---p 00000000 00:00 0
7f795cffb000-7f795d7fb000 rwxp 00000000 00:00 0 [stack:397845]
7f795d7fb000-7f795d7fc000 ---p 00000000 00:00 0
7f795d7fc000-7f795dffc000 rwxp 00000000 00:00 0 [stack:397844]
7f795dffc000-7f795dffd000 ---p 00000000 00:00 0
7f795dffd000-7f795e7fd000 rwxp 00000000 00:00 0 [stack:397843]
7f795e7fd000-7f795e7fe000 ---p 00000000 00:00 0
7f795e7fe000-7f795effe000 rwxp 00000000 00:00 0 [stack:397842]
7f795effe000-7f795efff000 ---p 00000000 00:00 0 Fatal Python error: Aborted
Thread 0x00007f7d00ffd700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 293 in safe_cuda_api_call
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 610 in clear
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 599 in add_item
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 964 in core
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1268 in free
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1382 in deref
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 477 in split_image_for_cuda
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 385 in receive_image_for_image_process
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 243 in on_post
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 440 in execute
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 172 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/channel.py", line 356 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 85 in handler_thread
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d017fe700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 293 in safe_cuda_api_call
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 610 in clear
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 599 in add_item
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 964 in core
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1268 in free
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1382 in deref
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 477 in split_image_for_cuda
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 385 in receive_image_for_image_process
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 243 in on_post
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 440 in execute
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 172 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/channel.py", line 356 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 85 in handler_thread
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d01fff700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 293 in safe_cuda_api_call
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 756 in allocator
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 741 in _attempt_allocation
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 758 in memalloc
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/devicearray.py", line 102 in __init__
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/devicearray.py", line 631 in from_array_like
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/devicearray.py", line 693 in auto_device
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/api.py", line 110 in to_device
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/devices.py", line 225 in _require_cuda_context
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 786 in cuda_process_roi_extract
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 477 in split_image_for_cuda
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 385 in receive_image_for_image_process
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 243 in on_post
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 440 in execute
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 172 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/channel.py", line 356 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 85 in handler_thread
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d14f8f700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 293 in safe_cuda_api_call
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 610 in clear
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 599 in add_item
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 964 in core
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1268 in free
File "/opt/anaconda3/lib/python3.6/site-packages/numba/cuda/cudadrv/driver.py", line 1382 in deref
File "/opt/anaconda3/lib/python3.6/site-packages/numba/utils.py", line 669 in __call__
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 477 in split_image_for_cuda
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 385 in receive_image_for_image_process
File "/home/admin/Downloads/falcon/SCRIPTS/cuda_roi_image_process_normalize_writeimage_api2.py", line 243 in on_post
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 440 in execute
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 172 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/channel.py", line 356 in service
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/task.py", line 85 in handler_thread
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d177fe700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/multiprocessing/connection.py", line 379 in _recv
File "/opt/anaconda3/lib/python3.6/multiprocessing/connection.py", line 407 in _recv_bytes
File "/opt/anaconda3/lib/python3.6/multiprocessing/connection.py", line 250 in recv
File "/opt/anaconda3/lib/python3.6/multiprocessing/pool.py", line 463 in _handle_results
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d17fff700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/threading.py", line 295 in wait
File "/opt/anaconda3/lib/python3.6/queue.py", line 164 in get
File "/opt/anaconda3/lib/python3.6/multiprocessing/pool.py", line 415 in _handle_tasks
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d1c833700 (most recent call first):
File "/opt/anaconda3/lib/python3.6/multiprocessing/pool.py", line 406 in _handle_workers
File "/opt/anaconda3/lib/python3.6/threading.py", line 864 in run
File "/opt/anaconda3/lib/python3.6/threading.py", line 916 in _bootstrap_inner
File "/opt/anaconda3/lib/python3.6/threading.py", line 884 in _bootstrap
Thread 0x00007f7d48b5f740 (most recent call first):
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/wasyncore.py", line 152 in poll
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/wasyncore.py", line 222 in loop
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/server.py", line 323 in run
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/__init__.py", line 17 in serve
File "/opt/anaconda3/lib/python3.6/site-packages/waitress/runner.py", line 279 in run
File "/opt/anaconda3/bin/waitress-serve", line 10 in <module>
Aborted (core dumped)