Posts: 39
Threads: 8
Joined: Jan 2022
Mar-31-2022, 04:00 AM
(This post was last modified: Mar-31-2022, 04:01 AM by GiggsB.)
Hi,
I have this python script where I am getting data from microcontroller (MCU) and calculating the CRC value of the received data. As the data is received, I am storing it into a new file with the name as DATE.T.HOUR.MINUTE.SECOND. After all data is received (~1 MB data), I am calculating the CRC value and comparing this with what MCU has sent. If the value matches, then good....if it does not matches, I plan to overwrite the data on the same file.
Can someone help me with how can I overwrite the data on the same file? Thanks.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 |
import datetime
import os
import struct
import time
import pigpio
import spidev
import zlib
bus = 0
device = 0
spi = spidev.SpiDev()
spi.open(bus, device)
spi.max_speed_hz = 4000000
spi.mode = 0
pi.set_mode(25, pigpio.INPUT)
rpi_crc=0
def output_file_path():
return os.path.join(os.path.dirname(__file__),
datetime.datetime.now().strftime("%dT%H.%M.%S") + ".csv")
def spi_process(gpio,level,tick):
data = bytes([0]*2048)
crc_data = bytes([0]*4)
spi.xfer2([0x02])
rpi_crc=0
with open(output_file_path(), 'w') as f:
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
rpi_crc = zlib.crc32(bytes(recv),rpi_crc)
f.write("\n")
f.write("\n".join([str(x) for x in values]))
mcu_crc_bytes = spi.xfer2(crc_data)
mcu_crc = struct.unpack("<"+"I"*1,bytes(mcu_crc_bytes))
mcu_crc_int = int(''.join(map(str,mcu_crc)))
if (rpi_crc != mcu_crc_int):
spi.xfer([0x03])
print("CRC did not match!!")
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
f.write("\n")
f.write("\n".join([str(x) for x in values]))
else:
print("CRC matched!!")
spi.xfer([0x04])
input("Press Enter to start the process ")
spi.xfer2([0x01])
cb1=pi.callback(INTERRUPT_GPIO, pigpio.RISING_EDGE, spi_process)
while True:
time.sleep(1)
|
Posts: 6,825
Threads: 20
Joined: Feb 2020
You really can't overwrite data in a file. You can append to a file, but any other kind of modification is performed by rewriting the file.
To rewrite a file you just open it for writing ("w"). This resets the file to empty. If the file is already open for writing you can use truncate(). I've never seen truncate() used, but you can read about it here.
https://docs.python.org/3/library/io.html
Posts: 39
Threads: 8
Joined: Jan 2022
(Mar-31-2022, 04:09 AM)deanhystad Wrote: .................
To rewrite a file you just open it for writing ("w"). This resets the file to empty. If the file is already open for writing you can use truncate(). I've never seen truncate() used, but you can read about it here. Thanks,
I changed my code to something like this (change line 38-48) in original posted code:
1 2 3 4 5 6 7 8 9 10 |
if (rpi_crc != mcu_crc_int):
spi.xfer([0x03])
print("NOOOOOOOOOOOOOOOO")
f.seek(0)
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
f.write("\n")
f.write("\n".join([str(x) for x in values]))
|
I have a doubt here, if I don't add the truncate and/or only seek(0), the size of the file doubles, basically the data is getting written after the last line where previously the data was stored. In both cases, if I only use seek or both truncate and seek, the size of the file remains same as previously.
I can't check the data for some reason if both produce same result but if anyone knows, would using either the seek only or both seek and truncate produce same results/do the same work?
Thanks.
Posts: 6,825
Threads: 20
Joined: Feb 2020
Mar-31-2022, 09:16 PM
(This post was last modified: Mar-31-2022, 09:16 PM by deanhystad.)
How are you opening your file? If you open a file for writing (mode = "w") it automatically truncates the file to zero. If you open a file with the mode set to append ("a") or read+write ("r+") this does not happen. Nor does it happen if you don't close and reopen the file. That is what's happening in your case when you call this:
1 2 3 4 5 6 |
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
f.write("\n")
f.write("\n".join([str(x) for x in values]))
|
Thinking more about your problem, I would not open the file at all unless the CRC checks out. This should make it easier to write your program.
Oh, and instead of this:
1 2 |
f.write("\n")
f.write("\n".join([str(x) for x in values]))
|
Do this
Posts: 39
Threads: 8
Joined: Jan 2022
(Mar-31-2022, 09:16 PM)deanhystad Wrote: How are you opening your file? Hi,
The file is already opened in "w" mode, I don't close it again or open it again. I have added my full code here:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 |
import datetime
import os
import struct
import time
import pigpio
import spidev
import zlib
bus = 0
device = 0
spi = spidev.SpiDev()
spi.open(bus, device)
spi.max_speed_hz = 4000000
spi.mode = 0
pi.set_mode(25, pigpio.INPUT)
rpi_crc=0
def output_file_path():
return os.path.join(os.path.dirname(__file__),
datetime.datetime.now().strftime("%dT%H.%M.%S") + ".csv")
def spi_process(gpio,level,tick):
data = bytes([0]*2048)
crc_data = bytes([0]*4)
spi.xfer2([0x02])
rpi_crc=0
with open(output_file_path(), 'w') as f:
t1=datetime.datetime.now()
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
rpi_crc = zlib.crc32(bytes(recv),rpi_crc)
f.write("\n")
f.write("\n".join([str(x) for x in values]))
mcu_crc_bytes = spi.xfer2(crc_data)
mcu_crc = struct.unpack("<"+"I"*1,bytes(mcu_crc_bytes))
mcu_crc_int = int(''.join(map(str,mcu_crc)))
if (rpi_crc != mcu_crc_int):
spi.xfer([0x03])
f.seek(0)
for x in range(392):
recv = spi.xfer2(data)
values = struct.unpack("<" +"I"*512, bytes(recv))
f.write("\n")
f.write("\n".join([str(x) for x in values]))
else:
spi.xfer([0x04])
t2=datetime.datetime.now()
print(t2-t1)
input("Press Enter to start the process ")
spi.xfer2([0x01])
cb1=pi.callback(INTERRUPT_GPIO, pigpio.RISING_EDGE, spi_process)
while True:
time.sleep(1)
|
So based on what you mentioned, I don't need to write the f.truncate().Right?
And thanks for the suggestions, I will implement in in my code and check it.
|
