Python Forum
how do i get utf-8 encoding for real?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how do i get utf-8 encoding for real?
#1
i am running into these errors and wondering how i change the encoding to really get utf-8.
Output:
lt1/pdh /home/pdh 46> py2 new-sessionin.py foo -141 ... local var: 'c' = '1' ... local var: 'n' = 141 ... local var: 'data' = ['\x8d'] Traceback (most recent call last):   File "new-sessionin.py", line 476, in <module>     result=main(sys.argv)   File "new-sessionin.py", line 439, in main     rr = decode_control( s[1:] )   File "new-sessionin.py", line 330, in decode_control     istr = [chr(x) for x in bytearray(istr,encoding='utf8')] UnicodeDecodeError: 'ascii' codec can't decode byte 0x8d in position 0: ordinal not in range(128) lt1/pdh /home/pdh 47> py3 new-sessionin.py foo -141 ... local var: 'c' = '1' ... local var: 'n' = 141 ... local var: 'data' = ['\x8d'] ... local var: 'rr' = (0, '\xc2\x8d') Traceback (most recent call last):   File "new-sessionin.py", line 476, in <module>     result=main(sys.argv)   File "new-sessionin.py", line 467, in main     r = execvp(c[0],c)   File "/usr/local/bin/../../lib/python3.5/os.py", line 615, in execvp     _execvpe(file, args)   File "/usr/local/bin/../../lib/python3.5/os.py", line 650, in _execvpe     exec_func(fullname, *argrest) UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128) lt1/pdh /home/pdh 48>
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#2
I have mention before that there most be something wrong with you Terminal/shell encoding.
Here some test you can do.
I use Mint 18.1 with Python 3.6.2.
If see LANG=C or similar, Python will use an ASCII encoder.
mint@mint ~ $ locale
LANG=en_US.UTF-8
LANGUAGE=
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
mint@mint ~ $ python
Python 3.6.2 (default, Aug  6 2017, 12:55:04) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import locale
>>> locale.getdefaultlocale()
('en_US', 'UTF-8')
>>> locale.getpreferredencoding()
'UTF-8'
>>> exit()
mint@mint ~ $ python -c "import sys; print(sys.stdout.encoding)"
UTF-8
mint@mint ~ $ python -c "print('Spicy jalapeño ☂')"
Spicy jalapeño ☂
Reply
#3
do you see anything you think i should change ...
CLUTTER_IM_MODULE=xim
COLS=184
COLUMNS=184
COMPIZ_BIN_PATH=/usr/bin/
COMPIZ_CONFIG_PROFILE=ubuntu
DBUS_SESSION_BUS_ADDRESS=unix:abstract=/tmp/dbus-4ZauRcxZWx
DEFAULTS_PATH=/usr/share/gconf/ubuntu.default.path
DESKTOP_SESSION=ubuntu
DISPLAY=:6
EDITOR=/usr/bin/emacs
EMAILADDR=############
EMAILDOMAIN=########
EMAILHOST=########
EMAILUSER=###
GDMSESSION=ubuntu
GDM_LANG=en_US
GNOME_DESKTOP_SESSION_ID=this-is-deprecated
GNOME_KEYRING_CONTROL=
GNOME_KEYRING_PID=
GPG_AGENT_INFO=/home/pdh/.gnupg/S.gpg-agent:0:1
GTK2_MODULES=overlay-scrollbar
GTK_IM_MODULE=ibus
GTK_MODULES=gail:atk-bridge:unity-gtk-module
HISTDIRECTORY=/home/pdh/.bash_history.d
HISTFILE=/home/pdh/.bash_history.d/2017-12-03-200309-11446-lt1-pdh-pts_71
HOME=/home/pdh
HOST=lt1
HOSTNAME=lt1
IM_CONFIG_PHASE=1
INSTANCE=
JAVA_HOME=/usr/lib/jvm/java-6-openjdk/bin/java
JOB=unity-settings-daemon
LANG=en_US.utf8
LANGUAGE=en_US.utf8
LC_ALL=C
LC_COLLATE=C
LC_CTYPE=C
LC_MESSAGES=C
LC_NUMERIC=C
LESS=-MX
LESSHISTFILE=-
LINES=50
LOGNAME=pdh
LS_OPTIONS=-b -F -T 0 --color=never
MANDATORY_PATH=/usr/share/gconf/ubuntu.mandatory.path
MANPATH=/usr/local/man:/usr/share/man:/usr/man:/usr/X11R6/man:/opt/kde/man
MINICOM=-c on
OLDPWD=/home/pdh
PATH=/home/pdh/.bin:/home/pdh/.cmd:/home/pdh/bin:/home/pdh/cmd:/usr/local/cmd:/usr/local/bin:/usr/bin:/bin:/usr/X11R6/bin:/opt/kde/bin
PROMPT_COMMAND=history -w ${HISTFILE}_tmp;mv -f ${HISTFILE}_tmp ${HISTFILE};export PS1="\h/\u `exec pwd` "'\!> '
PS1=\h/\u /home/pdh \!>
PWD=/home/pdh
QT4_IM_MODULE=xim
QT_ACCESSIBILITY=1
QT_IM_MODULE=ibus
QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1
QT_QPA_PLATFORMTHEME=appmenu-qt5
ROWS=50
SESSION=ubuntu
SESSIONTYPE=gnome-session
SESSION_MANAGER=local/lt1:@/tmp/.ICE-unix/7504,unix/lt1:/tmp/.ICE-unix/7504
SESSION_START=2017-12-03-200309
SESSION_TTY=pts_71
SESSION_UNIQUE=2017-12-03-200309-11446-lt1-pdh-pts_71
SHELL=/bin/bash
SHLVL=1
SSH_AUTH_SOCK=/run/user/1002/keyring/ssh
TERM=xterm-256color
TIMEFORMAT=[[ %0lR real %3R - user %3U - sys %3S - %P%% ]]
TTY=/dev/pts/71
TTYNAME=pts/71
UPSTART_EVENTS=xsession started
UPSTART_INSTANCE=
UPSTART_JOB=unity7
UPSTART_SESSION=unix:abstract=/com/ubuntu/upstart-session/1002/7259
USER=pdh
VISUAL=/usr/bin/emacs
VTE_VERSION=4205
WINDOWID=56623163
XAUTHORITY=/home/pdh/.Xauthority
XDG_CONFIG_DIRS=/etc/xdg/xdg-ubuntu:/usr/share/upstart/xdg:/etc/xdg
XDG_CURRENT_DESKTOP=Unity
XDG_DATA_DIRS=/usr/share/ubuntu:/usr/share/gnome:/usr/local/share:/usr/share:/var/lib/snapd/desktop:/var/lib/snapd/desktop
XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/pdh
XDG_MENU_PREFIX=gnome-
XDG_RUNTIME_DIR=/run/user/1002
XDG_SEAT=seat0
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
XDG_SESSION_DESKTOP=ubuntu
XDG_SESSION_ID=c7
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session6
XDG_SESSION_TYPE=x11
XDG_VTNR=13
XMODIFIERS=@im=ibus
_=/usr/bin/env
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply
#4
If you run all my test,dos it fail on the last one?
Reply
#5
the last one fails because that last character inside the single quotes triggers some kind of terminal input hotkey editing action (maybe in GNU readline).
Tradition is peer pressure from dead people

What do you call someone who speaks three languages? Trilingual. Two languages? Bilingual. One language? American.
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020