Python Forum
Pigz inside python - Reading compressed .gz file much faster
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Pigz inside python - Reading compressed .gz file much faster
#2
you can use python magic
Although this module is in PyPi,  the name conflicts with other packages of the same name, so you have to download and install the wheel
To do this:
  • Get the wheel from PyPi as follows
  • go to:  https://pypi.python.org/pypi/python-magic/
  • Download the wheel file (Current version): python_magic-0.4.15-py2.py3-none-any.whl
  • change directory to one containing wheel
  • from command line, install with:
    pip install python_magic-0.4.15-py2.py3-none-any.whl

Once you have that package installed, use the following code to find file type:
def check_filetype(filename):
    f = magic.Magic(mime=True, uncompress=True, filename)
    return f.from_file(filename)
This will avoid having to load entire zip file.
It will return a string of type:
Output:
'text/plain'
See the documentation here: https://github.com/ahupp/python-magic
Reply


Messages In This Thread
RE: Pigz inside python - Reading compressed .gz file much faster - by Larz60+ - Dec-21-2017, 08:01 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Reading an ASCII text file and parsing data... oradba4u 2 217 Jun-08-2024, 12:41 AM
Last Post: oradba4u
Sad problems with reading csv file. MassiJames 3 819 Nov-16-2023, 03:41 PM
Last Post: snippsat
  Navigating file directories and paths inside Jupyter Notebook Mark17 5 941 Oct-29-2023, 12:40 PM
Last Post: Mark17
  Reading a file name fron a folder on my desktop Fiona 4 1,096 Aug-23-2023, 11:11 AM
Last Post: Axel_Erfurt
  Reading data from excel file –> process it >>then write to another excel output file Jennifer_Jone 0 1,240 Mar-14-2023, 07:59 PM
Last Post: Jennifer_Jone
  Reading a file JonWayn 3 1,207 Dec-30-2022, 10:18 AM
Last Post: ibreeden
  Reading Specific Rows In a CSV File finndude 3 1,080 Dec-13-2022, 03:19 PM
Last Post: finndude
  Excel file reading problem max70990 1 977 Dec-11-2022, 07:00 PM
Last Post: deanhystad
  Reading All The RAW Data Inside a PDF NBAComputerMan 4 1,495 Nov-30-2022, 10:54 PM
Last Post: Larz60+
  Replace columns indexes reading a XSLX file Larry1888 2 1,082 Nov-18-2022, 10:16 PM
Last Post: Pedroski55

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020