Regular Expressions - so close yet so far

[bigpapa]

Hi all,

So I have this code below that filters the weight column in a data frame. And I want to remove the : and ; in-front of the number:

for entry in df.loc[df["Weight"]
                    .str.replace("\s", '', regex = True)
                    .str.contains('Weight', case = False, na = False), "Weight"].sample(10, random_state=2):
    
    print(re.findall(r'(?<=weight).*?(?=kg)', 
                     re.sub("\s", "", entry).lower() 
                    )
         ) 

[':16.696,00']
[';16.981,44', ';13.672,10', ';16.981,44', ';16.981,44', ';16.235,86']
[':17.046,00']
[':18.345,00']
[':17.624,00']
[':17,063.00']
['6000.0000']
[':18.583,000']
[':18.520,00']
[';16.981,44']

Thus far have tried adding [;:] into the regular expression like so:

for entry in df.loc[df["Weight"]
                    .str.replace("\s", '', regex = True)
                    .str.contains('Weight', case = False, na = False), "Weight"].sample(10, random_state=2):
    
    print(re.findall(r'(?<=weight[:;]).*?(?=kg)', 
                     re.sub("\s", "", entry).lower() 
                    )
         ) 

But it returns this:

['16.696,00']
['16.981,44', '13.672,10', '16.981,44', '16.981,44', '16.235,86']
['17.046,00']
['18.345,00']
['17.624,00']
['17,063.00']
[]
['18.583,000']
['18.520,00']
['16.981,44']

Do you see how if the item does not have a : or a ; it gets deleted? How do I prevent this. Also! I have tried [:;?] and [:;*?]


Also if someone has any idea of how to fix the commas separating the numbers so they're consistent that would be an added bonus.

Thank you, regular expressions are hard

[bowlofred]

Your findall only matches when there is a "weight" followed by one character from [;:]. That character isn't optional. If it's missing, no match.

So you want it to be an optional match. Add a ? after the character class.

r'(?<=weight[:;]?).*?(?=kg)

[bigpapa]

Your findall only matches when there is a "weight" followed by one character from [;:]. That character isn't optional. If it's missing, no match.

So you want it to be an optional match. Add a ? after the character class.

r'(?<=weight[:;]?).*?(?=kg)

Thanks for your reply, it still does the same thing :(

['16.696,00']
['16.981,44', '13.672,10', '16.981,44', '16.981,44', '16.235,86']
['17.046,00']
['18.345,00']
['17.624,00']
['17,063.00']
[]
['18.583,000']
['18.520,00']
['16.981,44']

[bowlofred]

I can only see your output, not what the regex is getting. For instance, if there is space between the weight and colon, that might explain things.

Can you show what the entry object is during the loop? At least a couple with and without the color or semicolon.

[bigpapa]

I can only see your output, not what the regex is getting. For instance, if there is space between the weight and colon, that might explain things.

Can you show what the entry object is during the loop? At least a couple with and without the color or semicolon.

Here is how some of the data is fed to regex. Let me know if you need more info! Thanks for your help with this.

WEIGHT: 18. 520, 0 0 KGS

WEIGHT: 18. 583, 000 KGS

WEIGHT 6000. 0000 KG

WEIGHT: 17. 624, 00 KGS

WEIGHT: 17. 046, 00 KGS

WEIGHT; 16. 235, 86 KGS

WEIGHT; 13. 672

WEIGHT: 29. 631, 000

WEIGHT: 218768. 000 KGS

WEIGHT: 15 MT

WEIGHT; 14. 834, 32 KGS

WEIGHT; 11. 311, 08 KG

[bowlofred]

Oh, right. Lookbehind patterns must be fixed-width. Can't put an optional piece in them. In that case I'd just use a regular search and capture the number.

import re

text = """WEIGHT: 18. 520, 0 0 KGS
WEIGHT: 18. 583, 000 KGS
WEIGHT 6000. 0000 KG
WEIGHT: 17. 624, 00 KGS
WEIGHT: 17. 046, 00 KGS
WEIGHT; 16. 235, 86 KGS
WEIGHT; 13. 672
WEIGHT: 29. 631, 000
WEIGHT: 218768. 000 KGS
WEIGHT: 15 MT
WEIGHT; 14. 834, 32 KGS
WEIGHT; 11. 311, 08 KG"""

for entry in text.splitlines():
    entry = entry.replace(" ", "").lower()
    print(entry, end=" => ")

    if m := re.search('weight[;:]?([\d,.]+)kg', entry):
        print(m.group(1))
    else:
        print("NO MATCH")

Output:
weight:18.520,00kgs => 18.520,00
weight:18.583,000kgs => 18.583,000
weight6000.0000kg => 6000.0000
weight:17.624,00kgs => 17.624,00
weight:17.046,00kgs => 17.046,00
weight;16.235,86kgs => 16.235,86
weight;13.672 => NO MATCH
weight:29.631,000 => NO MATCH
weight:218768.000kgs => 218768.000
weight:15mt => NO MATCH
weight;14.834,32kgs => 14.834,32
weight;11.311,08kg => 11.311,08

The inconsistent separators is more annoying. If you had something like 18,000, how would you be sure which way to interpret?

Assuming you want a period as a decimal separator, you could look if a comma occurs later than a period in the string. If it does, translate them to each other. If you only have one of the two, I don't have a good suggestion.