Aug-31-2021, 05:51 PM
Hi Everyone,
Working on rebuilding my code into Cython but during the rebuild i got stuck on a part of the code
and cannot seem to get it at Cython speed.
Already vectorized but that does not help.
Also tried creating a separate functions:
what is going on and how can is solve it??
Full code file is saved as .pyx and compiled:
Working on rebuilding my code into Cython but during the rebuild i got stuck on a part of the code
and cannot seem to get it at Cython speed.
for prc_buy in range(1,90): for prc_sell in range(1,90): newarr[:,1] = newarr[:,5] * (prc_buy / prc) newarr[:,2] = np.where(newarr[:,0] < newarr[:,1], 1,0) newarr[:,3] = newarr[:,6] / (prc_sell / prc) newarr[:,4] = np.where(newarr[:,0] > newarr[:,3], 1,0)once the code hits the section with np.where it gets really slow. i'm talking (without) 0.05 to (with)1.2 seconds slow.
Already vectorized but that does not help.
Also tried creating a separate functions:
@cython.boundscheck(False) # turn off bounds-checking for entire function @cython.wraparound(False) # turn off negative index wrapping for entire function def position_smaller(np.ndarray[object, ndim=2] arr, sct1, sct2, upd): cdef int row cdef Py_ssize_t hmax = arr.shape[0] for row in range(hmax): if arr[row][sct1] < arr[row][sct2]: arr[row][upd] = 1 return arrwhich does the same but is way slower then np.where.
what is going on and how can is solve it??
Full code file is saved as .pyx and compiled:
#!python #cython: language_level=3 import time import csv import sys, os import numpy as np cimport numpy as np import cython @cython.boundscheck(False) # turn off bounds-checking for entire function @cython.wraparound(False) # turn off negative index wrapping for entire function @cython.cdivision(True) def processing(np.ndarray[object, ndim=1] dff_arr, min_plus_k, min_plus_v ,name, span_1, span_2, span_3): cdef Py_ssize_t arr_shape = dff_arr.shape[0] cdef np.ndarray newarr = np.zeros(shape=(arr_shape,7)) newarr[:,0] = dff_arr.astype(np.float64) cdef float amount = 1000.0 cdef int buy_cnt_close cdef long arr cdef float prc_buy = 0.0 cdef float prc_sell = 0.0 cdef float prc = 100.0 cdef float sell_close, sell_coins cdef float buy_close_price = 0.0 cdef float buy_coins = 0.0 cdef float sum = 0.0 cdef int range_2 = 100 cdef int range_1 = 100 cdef list fields = [] save_path = '/media/usbstick/SPAN/' file_name = "%s.csv"%(name) completeName = os.path.join(save_path, file_name) newarr[:,5] = np.maximum.accumulate(newarr[:,0]) newarr[:,6] = np.minimum.accumulate(newarr[:,0]) cdef list list_of_lists = [] for prc_buy in range(range_2): for prc_sell in range(range_1): buy_cnt_close = 0 newarr[:,1] = newarr[:,5] * prc_buy / prc newarr[:,2] = np.where(newarr[:,0] < newarr[:,1], 1.0,0.0) newarr[:,3] = newarr[:,6] / prc_sell / prc newarr[:,4] = np.where(newarr[:,0] > newarr[:,3], 1.0,0.0) for arr in range(arr_shape): if (buy_cnt_close == 0 and newarr[arr][4] == 1): buy_close_price = newarr[arr][0] buy_coins = amount / buy_close_price buy_cnt_close = 1 continue elif (buy_cnt_close == 1 and newarr[arr][2] == 1): sell_close = newarr[arr][0] sell_coins = (buy_coins * sell_close) - amount fields=['%s ,%s, %s, %s, %s, %s, %s, %s, %s, %s ,%s\n'%(str(min_plus_k), str(min_plus_v) ,str(span_1), str(span_2), str(span_3), str(prc_buy) ,str(prc_sell) ,str(buy_close_price) ,str(buy_coins), str(sell_close), str(sell_coins))] list_of_lists.append(fields) buy_cnt_close = 0 sum += sell_coins if (sum > 0): with open(completeName, "a+", newline="") as f: writer = csv.writer(f) writer.writerows(list_of_lists) list_of_lists = [] else: list_of_lists = [] newarr = None