Jan-16-2020, 08:12 AM
Battling with Prometheus, in the below code, my prometheus counters are not incrementing.
tracked it down to the working being in a multiprocess pool...
Neither FILE_GESTER_TIME nor FILE_GESTER_LINE_COUNT is incrementing when view at port 8000
tracked it down to the working being in a multiprocess pool...
Neither FILE_GESTER_TIME nor FILE_GESTER_LINE_COUNT is incrementing when view at port 8000
# ... from prometheus_client import start_http_server, Summary, Counter # Prometheus metrics FILE_GESTER_TIME = Summary('BSA_file_gester_worker', 'Time spent loading a file') FILE_GESTER_LINE_COUNT = Counter('BSA_file_gester_line_count', 'Running counter of lines loaded') @FILE_GESTER_TIME.time() def worker(filename, config_params): # threaded process to attack each database async # --- Do work -- # loop ... for file in files: # --- Do some more work --- FILE_GESTER_LINE_COUNT.inc() # end for file in files # end worker def main(): start_http_server(8000) pool = multiprocessing.Pool(multiprocessing.cpu_count()) for file in files: result.append(pool.apply_async(worker, (file, config_Params ))) pool.close() pool.join() # Sleep here until all workers are done