Python Forum

Hello,

For some reason, the input data that are in UTF8 in SQLite end up as Latin1 in the output file:

import gpxpy.gpx
import sqlite3

gpx = gpxpy.gpx.GPX()
con = sqlite3.connect('input.sqlite')

con.row_factory = sqlite3.Row
cur = con.cursor()
cur.execute("SELECT name,latitude,longitude FROM table1");
results = cur.fetchall()
for row in results:
	w1 = gpxpy.gpx.GPXWaypoint(row["latitude"],row["longitude"])
	w1.name = row["name"]
	gpx.waypoints.append(w1)
	print("Waypoint: ",w1.name)

with open("map.gpx", "w") as f:
    f.write( gpx.to_xml())

I'm pretty sure data in SQLIte are in UTF8 because a run of ".output test.csv ; SELECT …" shows data in UTF8, so it looks like the issue lies in the code above.

Am I missing a switch somewhere?

Thank you.

From Python built-in help (>>> help('open')):

Quote:open(file, mode='r', buffering=-1, encoding=None, errors=None, newline=None, closefd=True, opene
r=None)

/.../

encoding is the name of the encoding used to decode or encode the
file. This should only be used in text mode. The default encoding is
platform dependent, but any encoding supported by Python can be
passed.

One can check (and set) default encoding in locale:

import locale
locale.getlocale()

However, more robust is to specify encoding while writing into file:

with open("map.gpx", "w", encoding='UTF-8') as f:

You nailed it. Thank you very much.