Python Forum
Why does Python/gpx save data as Latin1? - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/Forum-Python-Coding)
+--- Forum: General Coding Help (https://python-forum.io/Forum-General-Coding-Help)
+--- Thread: Why does Python/gpx save data as Latin1? (/Thread-Why-does-Python-gpx-save-data-as-Latin1)



Why does Python/gpx save data as Latin1? - Winfried - Nov-11-2019

Hello,

For some reason, the input data that are in UTF8 in SQLite end up as Latin1 in the output file:

import gpxpy.gpx
import sqlite3

gpx = gpxpy.gpx.GPX()
con = sqlite3.connect('input.sqlite')

con.row_factory = sqlite3.Row
cur = con.cursor()
cur.execute("SELECT name,latitude,longitude FROM table1");
results = cur.fetchall()
for row in results:
	w1 = gpxpy.gpx.GPXWaypoint(row["latitude"],row["longitude"])
	w1.name = row["name"]
	gpx.waypoints.append(w1)
	print("Waypoint: ",w1.name)

with open("map.gpx", "w") as f:
    f.write( gpx.to_xml())
I'm pretty sure data in SQLIte are in UTF8 because a run of ".output test.csv ; SELECT …" shows data in UTF8, so it looks like the issue lies in the code above.

Am I missing a switch somewhere?

Thank you.


RE: Why does Python/gpx save data as Latin1? - perfringo - Nov-12-2019

From Python built-in help (>>> help('open')):

Quote:open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opene
r=None)

/.../

encoding is the name of the encoding used to decode or encode the
file. This should only be used in text mode. The default encoding is
platform dependent, but any encoding supported by Python can be
passed.

One can check (and set) default encoding in locale:

import locale
locale.getlocale()
However, more robust is to specify encoding while writing into file:

with open("map.gpx", "w", encoding='UTF-8') as f:



RE: Why does Python/gpx save data as Latin1? - Winfried - Nov-12-2019

You nailed it. Thank you very much.