Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 [Regex] Findall returns wrong number of hits
#1
Hello,

It's probably something obvious to seasoned Python programmers, but I can't figure out why re.findall() returns the wrong number of hits when using the following code :
import re,sys,locale

# Open file
f = open('input.gpx', 'r')

#OK
#strings = re.findall(r'<trk>', f.read())
#BAD!
strings = re.findall(r'<trk>.+?</trk>', f.read())

f.close()

if strings:
	#27 instead of 348!
	print "Number of items : ", len(strings)
The file contains some French characters. Could it be that accented characters in the input file are preventing Python from reading the whole file?

FWIW, I'm using Python 2.7.14 on Windows.

Thank you.
Quote
#2
post input.gpx justbeamit or sample of file content and wanted output.
(Aug-21-2018, 03:28 PM)Winfried Wrote: FWIW, I'm using Python 2.7.14 on Windows
You should be using Python 3.6/3.7 and pip installation under Windows
Then Unicode support is better,like eg for French characters.
Quote
#3
Thanks.

So I removed 2.7.14, installed 3.7.0*, re-ran the code, and… same error: Wrong number of hits:

import re,sys,locale

# Open file
f = open('input.gpx', 'r')
#GOOD strings = re.findall(r'<trk>', f.read())

#Only 27!
#BAD strings = re.findall(r'<trk>.+?</trk>', f.read())

#Only 27!
p = re.compile(r'(<trk>.+?</trk>)',re.MULTILINE)
strings = p.findall(f.read())

f.close()

if strings:
	#27 instead of 348!
	print("Number of items : " + str(len(strings)))
* Checked that "python -V" returns "Python 3.7.0"
Quote
#4
It's the same problem for us,we don't know how input.gpx look or your wanted output from file.
A normal approach is to always read file with Unicode with UTF-8.
with open('input.gpx', encoding='utf-8') as f:
    print(f.read())
Quote
#5
The code above does display the whole file, as expected.

Maybe it's some odd character stopping the regex module dead in its tracks.

If someone wants to give it a shot, here's the input file.

As a work-around, the gpxpy module works OK with my file.

https://ocefpaf.github.io/python4oceanog...08/18/gpx/

Thanks for your help.
Quote
#6
GPX files use XML namespaces.
Then can use lxml
from lxml import etree

NSMAP = {"gpx": "http://www.topografix.com/GPX/1/1"}
tree = etree.parse("input.gpx")
for elem in tree.findall("gpx:trk", namespaces=NSMAP):
    print(elem)
Output:
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e469e0> <Element {http://www.topografix.com/GPX/1/1}trk at 0x4e469b8> <Element {http://www.topografix.com/GPX/1/1}trk at 0x4e46990> <Element {http://www.topografix.com/GPX/1/1}trk at 0x4e46968> ....
Try to get something out.
>>> elem
<Element {http://www.topografix.com/GPX/1/1}trk at 0x4e73760>
>>> elem[0].tag
'{http://www.topografix.com/GPX/1/1}name'
>>> elem[0].text
'Viroflay - Vélizy - Arcueil'
Can also search PyPi for gpx parser.
Quote
#7
Thanks, good to know.

I guess the file wasn't clean enough for the regex module to handle.
Quote
#8
I couldn't find why the gpx module uses two "import" lines:

import gpxpy
import gpxpy.gpx
What's the difference?

===
Edit : Found it. It's a sub-module.

To learn about a module, launch Python, and do something like this:

>>> import gpxpy
>>> help(gpxpy)

>>> import gpxpy.gpx
>>> help(gpxpy.gpx)
Quote
#9
In case another newbie stumbles on the same issue…

Gpxpy probably offers useful features anyway, but the reason the regex above didn't work, is simply that two switches are needed: "re.MULTILINE|re.DOTALL".

p = re.compile('<trk>.+?</trk>',re.MULTILINE|re.DOTALL)
m = p.findall(inputfile)
for item in m:
    print(item)
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  python gives wrong string length and wrong character thienson30 2 120 Oct-15-2019, 08:54 PM
Last Post: Gribouillis
  Number bug, calculation is wrong d3fi 2 195 Aug-27-2019, 05:29 AM
Last Post: perfringo
  Correct number wrong position func. albry 5 583 Jan-11-2019, 04:01 PM
Last Post: Larz60+
  Issue with re.findall alinaveed786 8 748 Oct-20-2018, 09:28 AM
Last Post: volcano63
  Combining the regex into single findall syoung 0 537 May-28-2018, 10:11 AM
Last Post: syoung
  Print next dataset on list when user hits any key disruptfwd8 2 573 Apr-30-2018, 10:56 PM
Last Post: disruptfwd8
  Regex: How to say 'any number of characters of any type until x'? JoeB 2 693 Jan-24-2018, 03:30 PM
Last Post: Mekire
  unable to print the list when using re.findall() satyaneel 5 1,419 Sep-27-2017, 10:26 AM
Last Post: buran
  'soup.findAll()' help - Want to retrieve multiple attribute values MattRHope 1 8,407 Jul-29-2017, 11:35 PM
Last Post: metulburr

Forum Jump:


Users browsing this thread: 1 Guest(s)