for lin in openfile:

[Skaperen]

i do get ANSI_X3.4-1968 for sys.stdout.encoding and my locale is mixed:

Output:
lt1/forums /home/forums 1> locale
LANG=en_US.utf8
LANGUAGE=en_US.utf8
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C
lt1/forums /home/forums 2>

which of those needs to be changed?

if i make free scripts available that users try to run when they have the wrong encoding, what instructions should i give them? for Windows, for Linux, for BSD, for MAC? for IBM mainframe not running any of those?

or can i do:

sys.stdout.encoding = 'UTF-8'

??

[snippsat]

which of those needs to be changed?

LC_CTYPE most be "en_US.UTF-8"
You change all when you first configure this.
Can mess up mine reconfigure.
All okay:

mint@mint ~ $ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

mint@mint ~ $ python3 -c "print('Spicy jalapeño ☂')"
Spicy jalapeño ☂

mint@mint ~ $ python3 -c "import sys; print(sys.stdout.encoding)"
UTF-8

Mess up:

mint@mint ~ $ export LC_ALL=C
mint@mint ~ $ export LANG=C

mint@mint ~ $ locale
LANG=C
LANGUAGE=
LC_CTYPE="C"
LC_NUMERIC="C"
LC_TIME="C"
LC_COLLATE="C"
LC_MONETARY="C"
LC_MESSAGES="C"
LC_PAPER="C"
LC_NAME="C"
LC_ADDRESS="C"
LC_TELEPHONE="C"
LC_MEASUREMENT="C"
LC_IDENTIFICATION="C"
LC_ALL=C

mint@mint ~python3 -c "print('Spicy jalapeño ☂')"
Unable to decode the command from the command line:
UnicodeEncodeError: 'utf-8' codec can't encode characters in position 19-20: surrogates not allowed

Fix:

mint@mint ~ $ sudo locale-gen "en_US.UTF-8"
Generating locales (this might take a while)...
  en_US.UTF-8... done
Generation complete.

mint@mint ~ $ sudo dpkg-reconfigure locales
Generating locales...
  en_US.UTF-8... up-to-date
Generation complete

mint@mint ~ $ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

mint@mint ~ $ python3 -c "print('Spicy jalapeño ☂')"
Spicy jalapeño ☂

mint@mint ~ $ python3 -c "import sys; print(sys.stdout.encoding)"
UTF-8

This should work for Debian-based distributions,other distro can search change locale language setting.

[Skaperen]

specifically, what changes do i need to make one time, each time i reboot, each time i login, and each time i start a new shell session? i already found a bug in a bash function my .bashrc script was using that was causing a bunch of LC_ variables to have the value "C". that bug has been there at least 10 years and thought i had them set to en_US.utf-8 back in 2010.

[snippsat]

specifically, what changes do i need to make one time, each time i reboot, each time i login, and each time i start a new shell session?

When you set values as i do,the changes is written permanently.

sudo locale-gen "en_US.UTF-8"
sudo dpkg-reconfigure locales

Debian-based distributions takes value from locale.gen.

mint@mint /etc $ cat locale.gen
en_US.UTF-8 UTF-8

A look at default.

mint@mint /etc/default $ cat locale
LANG="en_US.UTF-8"

mint@mint / $ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

[Skaperen]

i inherited my .bashrc from a non-Debian distro where a number of environment variable beginning with "LC_" were originally set to "C". i tried changing them a few years ago which mostly had no effect due to a bug which is fixed now. changing just LC_ALL to "$LANG" which was already "en_US.utf-8" fixed the issues.