Python Forum
How I can speed up my Cython module?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How I can speed up my Cython module?
#1
Python 3.7.5rc1, Cython 0.29.13

Hello, I created an ndarray in which, when reading a file line by line, I accumulate the first and last transaction price in the context of clients:
dtype=np.dtype([('client','U13'), ('day_begin','u4'), ('day_end','u4'), ('price_begin','f4'), ('price_end','f4')])   
return np.empty(0, dtype=dtype)
My Cython module, in which I get the lines from the csv- file and divide them into fields and convert them to the desired values to accumulate transaction statistics:
import numpy as np
cimport numpy as np

cpdef process_string(str file_str, str mask_client, str code, int end_day_int, np.ndarray periods_clients, dict line_by_client):
    if not file_str:
        return False, periods_clients
    
    cdef list fields = file_str.split(';')
    
    cdef str client = fields[3]
    if client.find(mask_client)==-1 and client!=code:
        return False, periods_clients
      
    cdef int current_date = _get_date_from_str(fields[1])
    if current_date > end_day_int:
        return True, periods_clients
    
    cdef double current_price = _convert_price_to_float(fields[5]) 
    periods_clients = _add_data_to_array(client, current_date, current_price, periods_clients, line_by_client)
    
    return False, periods_clients

cdef double _convert_price_to_float(str price):
    return np.float64(price.replace(',', '.'))
 
cdef int _get_date_from_str(str date_str):
    return 10000*int(date_str[6:10]) + 100*int(date_str[3:5]) + int(date_str[0:2])
    
cdef _add_data_to_array(str client, int current_date, double current_price, np.ndarray array, dict line_by_client):
        index_line = line_by_client.setdefault(client)
        if index_line is None:
            new_array = np.zeros(1, array.dtype)
            line = new_array[0]
            line['client'] = client
            line['day_begin'] = current_date
            line['price_begin'] = current_price
            line['day_end'] = current_date
            line['price_end'] = current_price
             
            array = np.append(array, new_array)
            line_by_client[client] = len(array)-1
             
        else:
            array[index_line]['day_end'] = current_date
            array[index_line]['price_end'] = current_price
            
        return array
Cython version works almost at the same speed as the python one. How I can speed up this module?
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Cython, Pandas, and Chained Assignment sawtooth500 4 262 Apr-13-2024, 04:18 AM
Last Post: sawtooth500
  I need to add data types to cython conversion python to c Good_AI_User 1 1,019 Aug-19-2022, 07:52 AM
Last Post: Gribouillis
  unable to load an image with C (Cython) HLD202 0 1,321 Jan-13-2022, 09:16 PM
Last Post: HLD202
  Cython np.where slows down the code Johanoosterwaal 1 1,742 Sep-01-2021, 07:33 AM
Last Post: Johanoosterwaal
  How can I use Cython to protect my Python Code? TurboC 2 4,146 Dec-03-2020, 10:37 AM
Last Post: padma121
  Python module speed or python speed in general Enrique6 1 1,853 May-04-2020, 06:21 PM
Last Post: micseydel
  Why does Cython generates bad code for Gentoo? AlekseyPython 1 1,854 Dec-26-2019, 01:57 PM
Last Post: AlekseyPython
  Error in compiling cython extension on Python 3.6.4 on Windows 8 goldenmean 3 5,786 Jun-05-2019, 09:37 PM
Last Post: Larz60+
  cython does not work skorost5 5 4,271 Dec-19-2018, 09:23 AM
Last Post: skorost5
  Help with building Cython modules. jarrod0987 2 3,334 Nov-15-2018, 03:57 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020