How can I solve this file handling issue?

[stevendaprano]

bit shifting operation was faster than struct.unpack()

Showing us the results without showing us the code that generated the results is pointless.

Let's do this the right way, using the timeit module

import random
from timeit import Timer
data = [random.randint(0, 255) for i in range(2048)]

t1 = Timer('x = struct.unpack(">" +"I"*512, bytes(data))', setup="import struct; from __main__ import data")
t2 = Timer('''
x = []
for y in range(0, 2048, 4):
    x.append(data[y]<<24 | data[y+1]<<16 | data[y+2]<<8 | data[y+3])
''', setup="from __main__ import data")

print("struct.unpack", min(t1.repeat(number=10000, repeat=7)))
print("manual loop with bitshift", min(t2.repeat(number=10000, repeat=7)))

On my computer, the result is not even close: struct.unpack is about seven times faster than the manual loop. On a Raspberry Pi, your results might be different -- but I would be shocked if the manual loop was faster. The unpack version does most of the work at the speed of C, while the manual loop is doing everything in Python.

In any case, its not really important. Even the slow manual loop version should be fast enough, less than a millisecond. If your code is taking 20 seconds to collect data from the Pi and write it to a file, the bottleneck making it slow is not the part where you convert the list of 8-bit ints to 32-bit ints.