+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: how do i get utf-8 encoding for real? (/thread-6680.html)
how do i get utf-8 encoding for real? - Skaperen - Dec-03-2017
i am running into these errors and wondering how i change the encoding to really get utf-8.
Output:lt1/pdh /home/pdh 46> py2 new-sessionin.py foo -141
... local var: 'c' = '1'
... local var: 'n' = 141
... local var: 'data' = ['\x8d']
Traceback (most recent call last):
File "new-sessionin.py", line 476, in <module>
result=main(sys.argv)
File "new-sessionin.py", line 439, in main
rr = decode_control( s[1:] )
File "new-sessionin.py", line 330, in decode_control
istr = [chr(x) for x in bytearray(istr,encoding='utf8')]
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8d in position 0: ordinal not in range(128)
lt1/pdh /home/pdh 47> py3 new-sessionin.py foo -141
... local var: 'c' = '1'
... local var: 'n' = 141
... local var: 'data' = ['\x8d']
... local var: 'rr' = (0, '\xc2\x8d')
Traceback (most recent call last):
File "new-sessionin.py", line 476, in <module>
result=main(sys.argv)
File "new-sessionin.py", line 467, in main
r = execvp(c[0],c)
File "/usr/local/bin/../../lib/python3.5/os.py", line 615, in execvp
_execvpe(file, args)
File "/usr/local/bin/../../lib/python3.5/os.py", line 650, in _execvpe
exec_func(fullname, *argrest)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)
lt1/pdh /home/pdh 48>RE: how do i get utf-8 encoding for real? - snippsat - Dec-03-2017
I have mention before that there most be something wrong with you Terminal/shell encoding.
Here some test you can do.
I use Mint 18.1 with Python 3.6.2.
If see LANG=C or similar, Python will use an ASCII encoder.
mint@mint ~ $ locale LANG=en_US.UTF-8 LANGUAGE= LC_CTYPE="en_US.UTF-8" LC_NUMERIC="en_US.UTF-8" LC_TIME="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_PAPER="en_US.UTF-8" LC_NAME="en_US.UTF-8" LC_ADDRESS="en_US.UTF-8" LC_TELEPHONE="en_US.UTF-8" LC_MEASUREMENT="en_US.UTF-8" LC_IDENTIFICATION="en_US.UTF-8" LC_ALL=
mint@mint ~ $ python
Python 3.6.2 (default, Aug 6 2017, 12:55:04)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF-8')
>>> locale.getpreferredencoding()
'UTF-8'
>>> exit()mint@mint ~ $ python -c "import sys; print(sys.stdout.encoding)"
UTF-8
mint@mint ~ $ python -c "print('Spicy jalapeño ☂')"
Spicy jalapeño ☂
RE: how do i get utf-8 encoding for real? - Skaperen - Dec-04-2017
do you see anything you think i should change ...
CLUTTER_IM_MODULE=xim
COLS=184
COLUMNS=184
COMPIZ_BIN_PATH=/usr/bin/
COMPIZ_CONFIG_PROFILE=ubuntu
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-4ZauRcxZWx
DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path
DESKTOP_SESSION=ubuntu
DISPLAY=:6
EDITOR=/usr/bin/emacs
EMAILADDR=############
EMAILDOMAIN=########
EMAILHOST=########
EMAILUSER=###
GDMSESSION=ubuntu
GDM_LANG=en_US
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
GNOME_KEYRING_CONTROL=
GNOME_KEYRING_PID=
GPG_AGENT_INFO=/home/pdh/.gnupg/S.gpg-agent:0:1
GTK2_MODULES=overlay-scrollbar
GTK_IM_MODULE=ibus
GTK_MODULES=gail:atk-bridge:unity-gtk-module
HISTDIRECTORY=/home/pdh/.bash_history.d
HISTFILE=/home/pdh/.bash_history.d/2017-12-03-200309-11446-lt1-pdh-pts_71
HOME=/home/pdh
HOST=lt1
HOSTNAME=lt1
IM_CONFIG_PHASE=1
INSTANCE=
JAVA_HOME=/usr/lib/jvm/java-6-openjdk/bin/java
JOB=unity-settings-daemon
LANG=en_US.utf8
LANGUAGE=en_US.utf8
LC_ALL=C
LC_COLLATE=C
LC_CTYPE=C
LC_MESSAGES=C
LC_NUMERIC=C
LESS=-MX
LESSHISTFILE=-
LINES=50
LOGNAME=pdh
LS_OPTIONS=-b -F -T 0 --color=never
MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path
MANPATH=/usr/local/man:/usr/share/man:/usr/man:/usr/X11R6/man:/opt/kde/man
MINICOM=-c on
OLDPWD=/home/pdh
PATH=/home/pdh/.bin:/home/pdh/.cmd:/home/pdh/bin:/home/pdh/cmd:/usr/local/cmd:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/opt/kde/bin
PROMPT_COMMAND=history -w ${HISTFILE}_tmp;mv -f ${HISTFILE}_tmp ${HISTFILE};export PS1="\h/\u `exec pwd` "'\!> '
PS1=\h/\u /home/pdh \!>
PWD=/home/pdh
QT4_IM_MODULE=xim
QT_ACCESSIBILITY=1
QT_IM_MODULE=ibus
QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1
QT_QPA_PLATFORMTHEME=appmenu-qt5
ROWS=50
SESSION=ubuntu
SESSIONTYPE=gnome-session
SESSION_MANAGER=local/lt1:@/tmp/.ICE-unix/7504,unix/lt1:/tmp/.ICE-unix/7504
SESSION_START=2017-12-03-200309
SESSION_TTY=pts_71
SESSION_UNIQUE=2017-12-03-200309-11446-lt1-pdh-pts_71
SHELL=/bin/bash
SHLVL=1
SSH_AUTH_SOCK=/run/user/1002/keyring/ssh
TERM=xterm-256color
TIMEFORMAT=[[ %0lR real %3R - user %3U - sys %3S - %P%% ]]
TTY=/dev/pts/71
TTYNAME=pts/71
UPSTART_EVENTS=xsession started
UPSTART_INSTANCE=
UPSTART_JOB=unity7
UPSTART_SESSION=unix:abstract=/com/ubuntu/upstart-session/1002/7259
USER=pdh
VISUAL=/usr/bin/emacs
VTE_VERSION=4205
WINDOWID=56623163
XAUTHORITY=/home/pdh/.Xauthority
XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/usr/share/upstart/xdg:/etc/xdg
XDG_CURRENT_DESKTOP=Unity
XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share:/usr/share:/var/lib/snapd/desktop:/var/lib/snapd/desktop
XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/pdh
XDG_MENU_PREFIX=gnome-
XDG_RUNTIME_DIR=/run/user/1002
XDG_SEAT=seat0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
XDG_SESSION_DESKTOP=ubuntu
XDG_SESSION_ID=c7
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session6
XDG_SESSION_TYPE=x11
XDG_VTNR=13
XMODIFIERS=@im=ibus
_=/usr/bin/env
RE: how do i get utf-8 encoding for real? - snippsat - Dec-04-2017
If you run all my test,dos it fail on the last one?
RE: how do i get utf-8 encoding for real? - Skaperen - Dec-06-2017
the last one fails because that last character inside the single quotes triggers some kind of terminal input hotkey editing action (maybe in GNU readline).