Python Forum
Not able to use boto library with compressed content in python3
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Not able to use boto library with compressed content in python3
#1
I am trying to use the boto library with python3 to upload the data to GCS buckets. My data is latin-1 encoded, but it fails with following exception
>>> get_utf8_value(zlib.compress(b'test').decode('latin-1'))b'x\xc2\x9c+I-.\x01\x00\x04]\x01\xc3\x81'>>> from boto.compat import StringIO>>> StringIO(get_utf8_value(zlib.compress(b'test').decode('latin-1')))Traceback (most recent call last): File "<stdin>", line 1, in <module>TypeError: initial_value must be str or None, not bytes
>>> from boto.utils import get_utf8_value
>>> get_utf8_value(zlib.compress(b'test').decode('latin-1'))
>>> from boto.compat import StringIO
>>> StringIO(get_utf8_value(zlib.compress(b'test').decode('latin-1')))
Traceback (most recent call last): File "<stdin>", line 1, in <module>TypeError: initial_value must be str or None, not bytes
Reply
#2
To be more specific I am using the latest version of boto library: 2.49.0 .
Please advice if some other version of boto library will be better suited for uploading the compressed strings (zlib compressed) to GCS Buckets? Or if some other APIs should be used?
I am resharing the code (from boto library)that gets invoked as part of invocation to: k.set_contents_from_string( zlib.compress(b'test').decode('latin-1'), headers={'Content-Type': "application/gzip"}) for uploading compressed content.

Quote:>>> from boto.utils import get_utf8_value
>>> get_utf8_value(zlib.compress(b'test').decode('latin-1'))
>>> from boto.compat import StringIO
>>> StringIO(get_utf8_value(zlib.compress(b'test').decode('latin-1')))
The exception thrown is Traceback (most recent call last): File "<stdin>", line 1, in <module>TypeError: initial_value must be str or None, not bytes
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Compressed multi page tiff wvanoeveren 2 2,611 Dec-28-2021, 11:40 AM
Last Post: Gribouillis
  MacOS BigSur Python3 - dyld: Library not loaded: trillionanswers 1 4,167 Mar-02-2021, 11:00 PM
Last Post: nilamo
  Gnuradio python3 is not compatible python3 xmlrpc library How Can I Fix İt ? muratoznnnn 3 4,823 Nov-07-2019, 05:47 PM
Last Post: DeaD_EyE
  Boto 3 and Mongo DB zatlas1 2 2,904 May-23-2019, 07:32 PM
Last Post: zatlas1
  How to know modules contained by python3 library sylas 3 5,145 Sep-04-2018, 01:40 PM
Last Post: sylas
  AWS ELB auto tagging with Lambda and boto 3 karteekdavid 4 3,579 Aug-14-2018, 03:26 AM
Last Post: karteekdavid
  How to remove html content from a column of the datafarme in Python3.6? PrateekG 3 3,116 Jul-26-2018, 08:18 AM
Last Post: Larz60+
  Issue python3.6 while inheriting telnet library sourabhjaiswal92 4 4,030 May-09-2018, 05:20 AM
Last Post: sourabhjaiswal92
  Pigz inside python - Reading compressed .gz file much faster jsmith7279 6 8,545 Dec-29-2017, 07:34 PM
Last Post: nilamo
  PyInstaller, how to create library folder instead of library.zip file ? harun2525 2 4,741 May-06-2017, 11:29 AM
Last Post: harun2525

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020