Python Forum
Help of multiple threads using cpython
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Help of multiple threads using cpython
#1
I recently tried to use a lot of threads (i.e. about 100) to run the same python function. That function will call an external library written in C. However, cpython will periodically throw an error at the line of calling the external C function, which looks like the following:

Error:
Exception in thread Thread-34: Traceback (most recent call last): File "/homes/liu1740/miniconda3/envs/my-rdkit-env/lib/python3.6/threading.py", line 916, in _bootstrap_inner self.run() File "/homes/liu1740/miniconda3/envs/my-rdkit-env/lib/python3.6/threading.py", line 864, in run self._target(*self._args, **self._kwargs) File "/u/subspace_s4/liu1740/Research/LocalGraphClustering/localgraphclustering/ncp.py", line 50, in ncp_node_worker ncpdata.results.extend(ncp_experiment(ncpdata, R, func, method_stats)) File "/u/subspace_s4/liu1740/Research/LocalGraphClustering/localgraphclustering/ncp.py", line 26, in ncp_experiment S = func(ncpdata.graph, R) File "<ipython-input-4-a67f0813539b>", line 1, in <lambda> funcs = {lambda G,R: lgc.spectral_clustering(G,R,alpha=alpha,rho=rho,method="acl")[0]:'acl;rho=%.0e'%(rho) for rho in rholist} File "/u/subspace_s4/liu1740/Research/LocalGraphClustering/localgraphclustering/spectral_clustering.py", line 96, in spectral_clustering rho = rho, epsilon = epsilon, method = method, ys = ys) File "/u/subspace_s4/liu1740/Research/LocalGraphClustering/localgraphclustering/approximate_PageRank.py", line 115, in approximate_PageRank alpha,rho,ref_nodes,iterations,G.lib) File "/u/subspace_s4/liu1740/Research/LocalGraphClustering/localgraphclustering/cpp/aclpagerank_cpp.py", line 52, in aclpagerank_cpp actual_length=fun(n,ai,aj,flag,alpha,eps,seedids,nseedids,maxsteps,xids,xlength,values) ctypes.ArgumentError: argument 11: <class 'TypeError'>: 'int' object is not callable
And it can happen at any argument randomly, not just argument 11. The code works fine with few threads. My guess is something is wrong during the conversion from a python variable to a C variable, but I am not sure and don't know how to fix it.
Reply


Messages In This Thread
Help of multiple threads using cpython - by LM19952012 - Jun-04-2018, 05:10 PM

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020