Python Forum
including big file contents in python source
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
including big file contents in python source
#1
i have a couple use cases where i need to include some big file contents. because these use cases are very complex and have too many things going on to sanely discuss them here, i will, instead, present a simplistic useless case; the kind of thing you might see in a classroom environment.
file (
here is that overly simplistic purpose. i need to create a python source file the is a means to transport a big file contents so that running the script "unpacks" the file to where the program is run.

the question i need to ask points to this simplistic purpose. what is the the best way to efficiently put a big (a few kilobytes to a few megabytes) file contents into python source code, so that, in the simplistic case, with included logic code, invoking the script can write that contents to a file.

my first thought is to have many lines of continued concatenated string with data in base 16, base 64, or base 85. any other ideas?
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
You can also use uuencode
import urllib.request, codecs
table = bytes.maketrans(b'\\"', b'xy')
invtable = bytes.maketrans(b'xy', b'\\"')

def to_literal(bdata):
    s = codecs.encode(bdata, 'uu').translate(table).decode()
    return 'b"""' + s + '"""'

def from_literal(data):
    s = eval(data).translate(invtable)
    return codecs.decode(s, 'uu')

s = urllib.request.urlopen('https://www.w3.org').read()
print(to_literal(s))
assert s == from_literal(to_literal(s))
Reply
#3
uuencode has single and double quotes in it. that can make it hard to use in source literals. but base85 doesn't have either.
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Algorithm for extracting comments from Python source code Pavel1982 6 512 Feb-28-2024, 09:52 PM
Last Post: Pavel1982
  Function to count words in a list up to and including Sam Oldman45 15 6,542 Sep-08-2023, 01:10 PM
Last Post: Pedroski55
  python move specific files from source to destination including duplicates mg24 3 1,095 Jan-21-2023, 04:21 AM
Last Post: deanhystad
  Python Snippets Source kucingkembar 0 630 Oct-18-2022, 12:50 AM
Last Post: kucingkembar
  Long-term stable source to get news headlines with Python? sandufi 4 1,914 Dec-23-2021, 09:48 AM
Last Post: sandufi
  Including data files in a package ChrisOfBristol 4 2,537 Oct-27-2021, 04:14 PM
Last Post: ChrisOfBristol
  Not including a constructor __init__ in the class definition... bytecrunch 3 11,807 Sep-02-2021, 04:40 AM
Last Post: deanhystad
  Navigating cpython source using python quazirfan 3 2,020 Aug-09-2021, 07:52 PM
Last Post: quazirfan
  how to create pythonic codes including for loop and if statement? aupres 1 1,922 Jan-02-2021, 06:10 AM
Last Post: Gribouillis
  Compiling Python 3.8.5 source code results in build error Deepan 0 2,175 Sep-14-2020, 04:11 AM
Last Post: Deepan

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020