i need to make a command that ...

[Skaperen]

i need to make a command that copies stdin to stdout and changes every non-printable character to an escape sequence ... a short one like \n if that character has one, or a hex one. this is so i can look a files that might have a few "strange" characters, to see what it really has, where.

[Gribouillis]

Here is one. You can improve it with argparse to include the --help option.

import sys

BUFSIZE = 1024

def main():
    while True:
        s = sys.stdin.read(BUFSIZE)
        if not s:
            break
        sys.stdout.write(repr(s)[1:-1])
        sys.stdout.flush()

if __name__ == '__main__':
    main()

[Skaperen]

i've done that. but repr() doesn't get this "right" in all cases. i know the range it gets "wrong" and could bypass it for those. "right" for what i need is what the Unix/C world does, not what repr() produces.

[Gribouillis]

what i need is what the Unix/C world does, not what repr() produces.

Then you need to give a full specification of how you want to represent the characters. Especially, how does Unix/C handle the representation of unicode characters in this context?

[ndc85430]

Is this something you really need to write? Is this not a job for tr, or sed?

[Skaperen]

tr can't do it. sed would be be very awkward. and so would awk. C could certainly do i but i'm committed to making things in Python so they are ready to go on any architecture.

the whole range from chr(7) to chr(13) are 2 character escape sequences. see man ascii on the nearest BSD or Linux system. 128 to 255 needs to be \xXX. 256 and beyond needs to come out as \Uxxxxxx to cover the whole Unicode (actually in UTF-8 in the file).