Python Forum

i have a large bytearray that i am writing to disk. i expect in most cases it will write the entire array in one call to binaryoutputfile.write(mybytearray). but, sometimes, there can be system conditions (it might be on a networked file system) that cause an incomplete write. simple code to do that could be:

# write the entire buffer
        while outbuf:
            rem=outfile.write(outbuf)
            if not rem:
                break
            outbuf[:-rem]=bytearray()

this code causes byte values in outbuf to be shifted by the slice assignment. it could be made to operate faster by calculating more numbers to slice out just what remains and not do a slice assignment. i don't have code for that, yet. but i am wondering if more complicated code to improve performance in rare cases could be considered reasonably pythonic.

note, i am not knowledgeable about how CPython has implemented slice assignments. i'm just imagining ways to do it in C.

This code looks a little strange to me because usually write() functions return the amount of data written (see for example io.BufferedWriter.write()). This data written should be at the beginning of outbuf and the code seems to chop the end of outbuf. Also the variable name 'rem' is misleading, it looks like 'remaining data' which is wrong.

Apart from that, you could perhaps write a separate function

def writeall(outfile, outbuf):
    ...

that would write all of outbuf into outfile, similarly to socket.sendall() vs socket.send(). Of course this function would read computed slices of outbuf without ever reassigning outbuf or one of its slices.

my aged memory must be corrupt and mixed that up with the system call. i like the distinct function idea. i will do that. i could even add that as a method for my zopen and ztopen classes.