clean up list elements and replace

clean up list elements and replace - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: clean up list elements and replace (/thread-11596.html)

clean up list elements and replace - metalray - Jul-17-2018

Dear Python Experts,

I am looking for a loop that goes through my elements in the list and removes
the comma , and ' ' signs. All other strings should remain and occurences like "March" be replaced by a number like 3/.
Is there a neat way to do it?

mylist = [[('','03/25/93','','','','','','','','','','','','','','','')],
[('', '', '', 'March 1976', '', '', '', '', '', '', '', '', '', '', '', '', '')]]

Exptected output:
mylist = [['03/25/93'],['3/1976']]

I would appreciate any help.

RE: clean up list elements and replace - ichabod801 - Jul-17-2018

We would appreciate you showing us what you had tried so far and explaining how it wasn't working. Then we would be glad to help.

RE: clean up list elements and replace - perfringo - Jul-17-2018

Is there only one date formatting ('monthname yearno') which must be converted? If so, following code should work (requires Python 3.6 or newer due to f-string):

import calendar                        # to convert month name to month num


my_new_list = []  
                     
for row in mylist: 
    my_new_list.append([])             
    dates = filter(str.strip, row[0])   # get rid of empty strings in rows tuple
    for date in dates:
        converted = date.split()
        if len(converted) == 2:
            monthnum = list(calendar.month_name).index(converted[0])
            my_new_list[-1].append(f'{monthnum}/{converted[1]}')
        else:
            my_new_list[-1].append(date)

Desired values should be in my_new_list

RE: clean up list elements and replace - metalray - Jul-25-2018

Hi perfringo,

thanks for your suggestion.

The dates are of various formats, I wish I could do date conversion in a second, seperate step.
Right now, what worries me most is the clean up.
I tried the following:

mylist = [[('','03/25/93','','','','','','','','','','','','','','','')],
        [('', '', '', 'March 1976', '', '', '', '', '', '', '', '', '', '', '', '', '')]]
mylist2 = [_ for _ in mylist if '' not in _]

#or

mylist2 = list(filter(lambda t: '' not in t, mylist))

No luck. Just doesnt work.

RE: clean up list elements and replace - snippsat - Jul-25-2018

(Jul-25-2018, 01:50 PM)metalray Wrote: Right now, what worries me most is the clean up.

>>> lst = [set(i[0]) for i in mylist]
>>> lst
[{'', '03/25/93'}, {'', 'March 1976'}]
>>> [i.discard('') for i in lst]
[None, None]
>>> my_list = [i.pop() for i in lst]
>>> my_list
['03/25/93', 'March 1976']

pip install python-dateutil

>>> from dateutil import parser
>>> 
>>> dt = parser.parse(my_list[1])
>>> dt
datetime.datetime(1976, 3, 25, 0, 0)
>>> print(f'Month is {dt.month} and year {dt.year}')
Month is 3 and year 1976

RE: clean up list elements and replace - perfringo - Jul-26-2018

(Jul-25-2018, 01:50 PM)metalray Wrote: Hi perfringo,

Right now, what worries me most is the clean up.

In snippet I provided there is row 8 which conveniently commented: # get rid of empty strings in rows tuple.

I walk you through some possible solutions to get rid of empty strings (written like in Python interactive interpreter).

Easiest way is to modify code in snippet provided, get rid parts of code where conversion is taking place:

>>> mylist = [[('','03/25/93','','','','','','','','','','','','','','','')],
... [('', '', '', 'March 1976', '', '', '', '', '', '', '', '', '', '', '', '', '')]]
>>> my_new_list = []
>>> for row in mylist:
...     my_new_list.append(filter(str.strip, row[0]))
...
>>> my_new_list
[<filter object at 0x105aac5c0>, <filter object at 0x105aac6a0>]

Filter objects are iterators and their location shown here are abritrary in specific computer. If you are not comfortable to work with filter objects / iterators (but you should) you can alternatively use list comprehension and do following:

>>> my_new_list = []
>>> for row in mylist:
...     my_new_list.append([date for date in row[0] if date])
...
>>> my_new_list
[['03/25/93'], ['March 1976]]

There is logical step from here. You can fit it all into one-liner with nested list comprehension:

>>> [[date for date in row[0] if date] for row in mylist]
[['03/25/93'], ['March 1976']]

This is datastructure you stated as expected output. I should mention, that this is not same as mylist structure as expected output is missing tuple level. I don't no whether is important or not.

Some words of unsolicited wisdom: you have to chew your food.

This means that if you ask for help and get one you should pay attention to that and try to modify ideas provided to suit your needs. No one will solve your problems but yourself. There are people out there who are willing to help you but not do your work for you.

RE: clean up list elements and replace - perfringo - Jul-26-2018

(Jul-25-2018, 01:50 PM)metalray Wrote: I tried the following:

mylist = [[('','03/25/93','','','','','','','','','','','','','','','')],
        [('', '', '', 'March 1976', '', '', '', '', '', '', '', '', '', '', '', '', '')]]
mylist2 = [_ for _ in mylist if '' not in _]

#or

mylist2 = list(filter(lambda t: '' not in t, mylist))

No luck. Just doesnt work.

You must ask yourself - what I want to accomplish?

What I would have done:
- goal: to get rid empty strings in datastructure (in tuples inside list of lists)

Steps:
- how to get rid empty strings in sequence/iterable
- how to wrap it so that it works on my datastructure

How to get rid of empty strings in iterable?

As suggested by PEP-8 Style Guide for Python Code >>> Programing recommendations: For sequences, (strings, lists, tuples), use the fact that empty sequences are false.

Empowered by this knowledge we can have (list) comprehension:

[date for date in mylist if date]

if date part of code checks whether string is true (non-empty).

First step accomplished, second to go.

Lets try to apply this comprehension on sequence which is inside list of lists (matrix). If order to do so we nest comprehension:

[[date for date in row if date] for row in mylist]

This is not it, but we are getting closer. We are accessing list inside list but there is tuple inside that list. Fortunately for us it appears that there is only one tuple so we can take advantage of using index:

[[date for date in row[0] if date] for row in mylist]

Step two accomplished, goal achieved.

Comprehension is very powerful tool but at times it hard to comprehend Smile

This simple example may help you crasp syntax:

[letter for word in sentence for letter in word]

In human readable language: for every word in sentence for every letter in word return letter.

Don't let 'letter for word' at the beginning to fool you. It is actually:

for word in sentence:
    for letter in word:

RE: clean up list elements and replace - metalray - Aug-30-2018

Wow, Thanks perfringo!
That is something to chew on :D