Remove empty keys in a python list

[python_student]

I have a list of dictionaries and in case if some dictionary has empty key I want to remove it completely from the list. In my example,
I need to remove all third block because column3 is empty.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

a = [     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": 98         }     },     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": 180         }     },     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": ""         }     }    ]

This is my code but is not removing and I think it's because of json structure

1 2 3 4

a = [d for d in payload if "" not in d.values()]   print(type(a)) <class 'list'>

Does anyone knows how can I solve it?

[BashBedlam]

I couldn't figure out how to do it with list comprehension but here's something that at least does the job.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37

a = [     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": 98         }     },     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": 180         }     },     {         "type": "some.type2",         "payload": {             "column1": "12345",             "column2": "description",             "column3": ""         }     }    ] z = []   for dictionary in a :     for value in dictionary ['payload'].values () :         if value == '' :             break     else :         z.append (dictionary)   for dictionary in z :     print (dictionary)

[python_student]

I have tried this solution, but did not work. Here is the code I have tried. I have "if conditional" from type because I have others type. This specially is with problem

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

print(type(a)) payload = json.loads(json.dumps(a))   if type(payload) == list:     payload2 = []     for dictionary in payload:         for value in dictionary['payload'].value():             if value =='':                 break             else:                 payload2.append(dictionary)     payload = payload2 else:     payload = {k:v for (k,v) in payload.items() if v != ""}     message['messages'] = payload print(json.dumps( message ))

Any idea?

[ibreeden]

Please elaborate "but did not work". Did you get an error message? Or an unexpected result? Or what.

[deanhystad]

The tricky part is to "flatten" the dictionary so you look at all the keys.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61

a = [     {         "type": "X",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": 1         }     },     {         "type": "X",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": 0         }     },     {         "type": "X",         "payload": {             "column1": "",             "column2": [1, 2, 3],             "column3": 1         }     },     {         "type": "X",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": None         }     },     {         "type": "X",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": {}         }     } ]   def empty_key(dictionary):     """Recursively scan dictionary for values that are empty collections or None.     Return associated key and value or None if no 'empty' values found."""     for key in dictionary.keys():         value = dictionary[key]         if value is None:             return key, value         elif isinstance(value, (list, tuple, set, str)) and not value:             return key, value         elif isinstance(value, dict):             if value:                 if (value := empty_key(value)):                      return value             else:                 return key, value  # Empty dictionary     return None   print([x for x in a if empty_key(x) is None])

This is easily expanded to not only scan dictionary values but list, tuple, and set values as well.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73

a = [     {         "type": "Not empty",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": 1         }     },     {         "type": "Zero, Not empty",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": 0         }     },     {         "type": "column1 is empty str",         "payload": {             "column1": "",             "column2": [1, 2, 3],             "column3": 1         }     },     {         "type": "column3 is None",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": None         }     },     {         "type": "column3 is empty dictionary",         "payload": {             "column1": "Y",             "column2": [1, 2, 3],             "column3": {}         }     },     {         "type": "Column2 has empty list in list",         "payload": {             "column1": "Y",             "column2": [1, 2, []],             "column3": {"A":1}         }     } ]   def isempty(thing):     """Recursively scan for emptyness.  Return True if emptyness is found     in the form of an empty string, list, tuple, set or dictionary."""     if isinstance(thing, str):         if not thing:             return True     elif isinstance(thing, (list, tuple, set)):         if not thing:             return True         for subthing in thing:             if isempty(subthing):                 return True     elif isinstance(thing, dict):         if not thing:             return True         for subthing in thing.values():             if isempty(subthing):                 return True     return False   for thing in [x for x in a if isempty(x)]:     print(thing)

Output:
{'type': 'column1 is empty str', 'payload': {'column1': '', 'column2': [1, 2, 3], 'column3': 1}}
{'type': 'column3 is None', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': None}}
{'type': 'column3 is empty dictionary', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': {}}}
{'type': 'Column2 has empty list in list', 'payload': {'column1': 'Y', 'column2': [1, 2, []], 'column3': {'A': 1}}}

[python_student]

Sorry to forget that

Error:
TypeError                                 Traceback (most recent call last)
<ipython-input-61-8adcda79951d> in <module>
    112     payload2 = []
    113     for i in range(len(payload)):
--> 114         for value in i['payload'].value():
    115             if value =='':
    116                 break

TypeError: 'int' object is not subscriptable

I had this error

[deanhystad]

That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.

1 2 3 4

if isinstance(payload, list):     payload = [d for d in payload if not "" in d["payload"].values()] else:     payload = {k:v for k, v in payload.items() if v != ""}

I still think a more generic solution is better.

[python_student]

That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.

1 2 3 4

if isinstance(payload, list):     payload = [d for d in payload if not "" in d["payload"].values()] else:     payload = {k:v for k, v in payload.items() if v != ""}

I still think a more generic solution is better.

Thank You