Python Forum
unicodeencodeerror 'charmap' codec can't encode characters in position
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
unicodeencodeerror 'charmap' codec can't encode characters in position
#1
The whole error is:
"unicodeencodeerror 'charmap' codec can't encode characters in position 1-6: character maps to <undefined>"

Appears when I run my program in the win console.

When I run it through the PyCharm IDE it does not appear.

The characters it's refering to, are non Latin characters.

I tried with .encode('utf-8') and .decode('utf-8') and seems ok.

Is this the 'proper' way to handle this?
Reply
#2
That is not the whole error, which has multiple lines.

Also please show a minimal running code example, that we can run to produce this error you see.
Reply
#3
By default, Windows does not use 'utf-8', it uses 'code pages'.  Your Pycharm settings most likely tells it to use 'utf-8', whereas your command terminal defaults to a code page.  The code page is determined by the locale you chose at Windows installation.  These code pages are based on standards created prior to the currently accepted 'utf-8' (which is a subset of Unicode).  When Windows does use Unicode, it is utf-16, when you want to use utf-8, Windows has a (rather crappy, imho) code page of 65001.

A list of all Python supported code pages can be found at:
Python main directory \Lib\encodings

A good read is here : Unicode
If it ain't broke, I just haven't gotten to it yet.
OS: Windows 10, openSuse 42.3, freeBSD 11, Raspian "Stretch"
Python 3.6.5, IDE: PyCharm 2018 Community Edition
Reply
#4
You should post an example and python version is important.
Quote:I tried with .encode('utf-8') and .decode('utf-8') and seems ok.
From where?

Use a better shell cmder.
In cmd:
C:\>which python
'which' is not recognized as an internal or external command,
operable program or batch file.

C:\>python -c "import sys; print(sys.stdout.encoding)"
utf-8

C:\>python -c "print('Spicy jalapeño ? ? ? ? ')"
Spicy jalapeño ? ? ? ? 

C:\>python -c "print('? ? ? ? ?')"
? ? ? ? ?
The same in cmder:
C:\
λ which python
/c/python36/python

C:\
λ python -c "import sys; print(sys.stdout.encoding)"
utf-8

C:\
λ python -c "print('Spicy jalapeño ☂ ☃ ★ ☁')"
Spicy jalapeño ☂ ☃ ★ ☁

C:\
λ python -c "print('♚ ✈ ☹ ☠ ☝')"
♚ ✈ ☹ ☠ ☝
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  encode/decode to show correct country letters in a CTk combobox janeik 2 710 Sep-02-2023, 09:46 AM
Last Post: janeik
Question UnicodeEncodeError: 'ascii' codec can't encode character u'\xe8' in position 562: ord ctrldan 23 4,804 Apr-24-2023, 03:40 PM
Last Post: ctrldan
  UnicodeDecodeError: 'utf-8' codec can't decode byte 0xd2 in position 16: invalid cont Melcu54 3 4,917 Mar-26-2023, 12:12 PM
Last Post: Gribouillis
  [SOLVED] [Debian] UnicodeEncodeError: 'ascii' codec Winfried 1 1,020 Nov-16-2022, 11:41 AM
Last Post: Winfried
  UnicodeDecodeError: 'charmap' codec can't decode byte 0x81 in position 34: character Melcu54 7 18,840 Sep-26-2022, 10:09 AM
Last Post: Melcu54
  UnicodeEncodeError - Dealing with Japanese Characters fioranosnake 2 2,442 Jul-07-2022, 08:43 PM
Last Post: fioranosnake
  UnicodeEncodeError: 'ascii' codec can't encode character '\xfd' in position 14: ordin Armandito 6 2,715 Apr-29-2022, 12:36 PM
Last Post: Armandito
  UnicodeEncodeError caused by print when program runs from Popen SheeppOSU 5 2,909 Jan-13-2022, 08:11 AM
Last Post: SheeppOSU
  [UnicodeEncodeError from smtplib] yoohooos 0 3,381 Sep-25-2021, 04:27 AM
Last Post: yoohooos
  ASCII-Codec in Python3 [SOLVED] AlphaInc 4 6,109 Jul-07-2021, 07:05 PM
Last Post: AlphaInc

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020