[Regex] Findall returns wrong number of hits

[Regex] Findall returns wrong number of hits - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: [Regex] Findall returns wrong number of hits (/thread-12359.html)

[Regex] Findall returns wrong number of hits - Winfried - Aug-21-2018

Hello,

It's probably something obvious to seasoned Python programmers, but I can't figure out why re.findall() returns the wrong number of hits when using the following code :

import re,sys,locale

# Open file
f = open('input.gpx', 'r')

#OK
#strings = re.findall(r'<trk>', f.read())
#BAD!
strings = re.findall(r'<trk>.+?</trk>', f.read())

f.close()

if strings:
	#27 instead of 348!
	print "Number of items : ", len(strings)

The file contains some French characters. Could it be that accented characters in the input file are preventing Python from reading the whole file?

FWIW, I'm using Python 2.7.14 on Windows.

Thank you.

RE: [Regex] Findall returns wrong number of hits - snippsat - Aug-21-2018

post input.gpx justbeamit or sample of file content and wanted output.

(Aug-21-2018, 03:28 PM)Winfried Wrote: FWIW, I'm using Python 2.7.14 on Windows

You should be using Python 3.6/3.7 and pip installation under Windows
Then Unicode support is better,like eg for French characters.

RE: [Regex] Findall returns wrong number of hits - Winfried - Aug-21-2018

Thanks.

So I removed 2.7.14, installed 3.7.0*, re-ran the code, and… same error: Wrong number of hits:

import re,sys,locale

# Open file
f = open('input.gpx', 'r')
#GOOD strings = re.findall(r'<trk>', f.read())

#Only 27!
#BAD strings = re.findall(r'<trk>.+?</trk>', f.read())

#Only 27!
p = re.compile(r'(<trk>.+?</trk>)',re.MULTILINE)
strings = p.findall(f.read())

f.close()

if strings:
	#27 instead of 348!
	print("Number of items : " + str(len(strings)))

* Checked that "python -V" returns "Python 3.7.0"

RE: [Regex] Findall returns wrong number of hits - snippsat - Aug-21-2018

It's the same problem for us,we don't know how input.gpx look or your wanted output from file.
A normal approach is to always read file with Unicode with UTF-8.

with open('input.gpx', encoding='utf-8') as f:
    print(f.read())

RE: [Regex] Findall returns wrong number of hits - Winfried - Aug-21-2018

The code above does display the whole file, as expected.

Maybe it's some odd character stopping the regex module dead in its tracks.

If someone wants to give it a shot, here's the input file.

As a work-around, the gpxpy module works OK with my file.

https://ocefpaf.github.io/python4oceanographers/blog/2014/08/18/gpx/

Thanks for your help.

RE: [Regex] Findall returns wrong number of hits - snippsat - Aug-21-2018

GPX files use XML namespaces.
Then can use lxml

from lxml import etree

NSMAP = {"gpx": "http://www.topografix.com/GPX/1/1"}
tree = etree.parse("input.gpx")
for elem in tree.findall("gpx:trk", namespaces=NSMAP):
    print(elem)

Output:<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e469e0>
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e469b8>
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e46990>
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e46968>
....

Try to get something out.

>>> elem
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e73760>
>>> elem[0].tag
'{http://www.topografix.com/GPX/1/1}name'
>>> elem[0].text
'Viroflay - Vélizy - Arcueil'

Can also search PyPi for gpx parser.

RE: [Regex] Findall returns wrong number of hits - Winfried - Aug-21-2018

Thanks, good to know.

I guess the file wasn't clean enough for the regex module to handle.

RE: [Regex] Findall returns wrong number of hits - Winfried - Aug-22-2018

I couldn't find why the gpx module uses two "import" lines:

import gpxpy
import gpxpy.gpx

What's the difference?

===
Edit : Found it. It's a sub-module.

To learn about a module, launch Python, and do something like this:

>>> import gpxpy
>>> help(gpxpy)

>>> import gpxpy.gpx
>>> help(gpxpy.gpx)

RE: [Regex] Findall returns wrong number of hits - Winfried - Aug-23-2018

In case another newbie stumbles on the same issue…

Gpxpy probably offers useful features anyway, but the reason the regex above didn't work, is simply that two switches are needed: "re.MULTILINE|re.DOTALL".

p = re.compile('<trk>.+?</trk>',re.MULTILINE|re.DOTALL)
m = p.findall(inputfile)
for item in m:
    print(item)