Jul-21-2019, 03:08 PM
Hello!
I am brand new, so bear with me. I am trying to do a specific polynomial of crc32 in py and that seems to be working but its horribly slow (c++ does 2 million of these crc32s + an md5 in 0.4 or less seconds, java did it in 1.5 sec, and python is taking 6-7 sec for just 1M crcs!!). My testing indicates that the shift (<< 24) operator is the main issue, and I tried division (it was no good) and am now looking at ctypes (I am proficient at c & c++) to see if I can just directly access the offending byte.
I want to take a ctypes pointer to a 4 byte integer and get the third byte (as unsigned byte).
I have been messing with the cast function and the pointers for a couple of hours now and can't seem to nail down the syntax (in spite of similar but not quite what I want examples online).
i = c_uint(12345678)
cast(i, POINTER(c_byte*4)())
Can anyone direct me, or am I going down the wrong approach, and if so, is there a fast way to do x >> 24 (and similar) in python? I know I can call my C code but we don't use it at my company, no one much knows it and its marked avoid at all costs (annoying, but that is how it is). I am not expecting the 0.4 but would love to get cut it on down to at least java's speeds.
I am brand new, so bear with me. I am trying to do a specific polynomial of crc32 in py and that seems to be working but its horribly slow (c++ does 2 million of these crc32s + an md5 in 0.4 or less seconds, java did it in 1.5 sec, and python is taking 6-7 sec for just 1M crcs!!). My testing indicates that the shift (<< 24) operator is the main issue, and I tried division (it was no good) and am now looking at ctypes (I am proficient at c & c++) to see if I can just directly access the offending byte.
I want to take a ctypes pointer to a 4 byte integer and get the third byte (as unsigned byte).
I have been messing with the cast function and the pointers for a couple of hours now and can't seem to nail down the syntax (in spite of similar but not quite what I want examples online).
i = c_uint(12345678)
cast(i, POINTER(c_byte*4)())
Can anyone direct me, or am I going down the wrong approach, and if so, is there a fast way to do x >> 24 (and similar) in python? I know I can call my C code but we don't use it at my company, no one much knows it and its marked avoid at all costs (annoying, but that is how it is). I am not expecting the 0.4 but would love to get cut it on down to at least java's speeds.