compressing a file

[Skaperen]

i have a text file. i want to read the file in a compressed form (in binary, of course). lzma.open() seems to do the opposite of what i need. it uncompresses but i need to do compression. how can i do compression and reading? i want to avoid compressing in memory.

[BashBedlam]

It is difficult to determine exactly what you are asking but the following code will read any file as bytes. Is this what you are looking for?

with open ('document.txt', 'rb') as input_file :
	the_file_data = input_file.read ()

[Skaperen]

using lzma.open() to read a file can only uncompress. to do compression, it must be used to write a file. i want to read a file and compress it so i end up with the compressed bytes.

alas, after some other design changes, that file no long is on disk. i will have it in memory., so, i can use lzma.LZMACompressor() to do it all in memory and end up with the compressed bytes.

[jefsummers]

So you want to read an uncompressed source file and compress it, but did not want the compression to be done in memory? Where did you want the compression to occur?

[bowlofred]

So you want to read an uncompressed source file and compress it, but did not want the compression to be done in memory? Where did you want the compression to occur?

I interpreted the question as wanting to compress it, but read the data as a stream so the entire file is not in memory at one time. The docs suggested to me that compressing a single object is simple, but if you wanted a stream compressor, that's not explicitly shown.

I would assume there's some way to chain the blockIO devices so you could read() the compression results (rather than direct it to a file on disk), but wasn't sure how to accomplish that.

[Skaperen]

So you want to read an uncompressed source file and compress it, but did not want the compression to be done in memory? Where did you want the compression to occur?

i did not want a solution that involved reading the whole file into memory and then compressing because it would take too much memory. i wanted to do incremental compression to use less memory. i've found a way to do it as the file is generated so i don't have to keep the uncompressed data.