Python Forum

I have a list of dictionaries and in case if some dictionary has empty key I want to remove it completely from the list. In my example,
I need to remove all third block because column3 is empty.

a = [
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 98
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 180
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": ""
        }
    }    
]

This is my code but is not removing and I think it's because of json structure

a = [d for d in payload if "" not in d.values()]

print(type(a))
<class 'list'>

Does anyone knows how can I solve it?

I couldn't figure out how to do it with list comprehension but here's something that at least does the job.

a = [
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 98
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 180
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": ""
        }
    }    
]
z = [] 

for dictionary in a :
	for value in dictionary ['payload'].values () :
		if value == '' :
			break
	else :
		z.append (dictionary)

for dictionary in z :
	print (dictionary)

I have tried this solution, but did not work. Here is the code I have tried. I have "if conditional" from type because I have others type. This specially is with problem

print(type(a))
payload = json.loads(json.dumps(a))

if type(payload) == list:
    payload2 = []
    for dictionary in payload:
        for value in dictionary['payload'].value():
            if value =='':
                break
            else:
                payload2.append(dictionary)
    payload = payload2
else:
    payload = {k:v for (k,v) in payload.items() if v != ""}


message['messages'] = payload
print(json.dumps( message ))

Any idea?

Please elaborate "but did not work". Did you get an error message? Or an unexpected result? Or what.

The tricky part is to "flatten" the dictionary so you look at all the keys.

a = [
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 0
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": None
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": {}
        }
    }
]

def empty_key(dictionary):
    """Recursively scan dictionary for values that are empty collections or None.
    Return associated key and value or None if no 'empty' values found."""
    for key in dictionary.keys():
        value = dictionary[key]
        if value is None:
            return key, value
        elif isinstance(value, (list, tuple, set, str)) and not value:
            return key, value
        elif isinstance(value, dict):
            if value:
                if (value := empty_key(value)):
                     return value
            else:
                return key, value  # Empty dictionary
    return None

print([x for x in a if empty_key(x) is None])

This is easily expanded to not only scan dictionary values but list, tuple, and set values as well.

a = [
    {
        "type": "Not empty",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "Zero, Not empty",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 0
        }
    },
    {
        "type": "column1 is empty str",
        "payload": {
            "column1": "",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "column3 is None",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": None
        }
    },
    {
        "type": "column3 is empty dictionary",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": {}
        }
    },
    {
        "type": "Column2 has empty list in list",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, []],
            "column3": {"A":1}
        }
    }
]

def isempty(thing):
    """Recursively scan for emptyness.  Return True if emptyness is found
    in the form of an empty string, list, tuple, set or dictionary."""
    if isinstance(thing, str):
        if not thing:
            return True
    elif isinstance(thing, (list, tuple, set)):
        if not thing:
            return True
        for subthing in thing:
            if isempty(subthing):
                return True
    elif isinstance(thing, dict):
        if not thing:
            return True
        for subthing in thing.values():
            if isempty(subthing):
                return True
    return False

for thing in [x for x in a if isempty(x)]:
    print(thing)

Output:
{'type': 'column1 is empty str', 'payload': {'column1': '', 'column2': [1, 2, 3], 'column3': 1}}
{'type': 'column3 is None', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': None}}
{'type': 'column3 is empty dictionary', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': {}}}
{'type': 'Column2 has empty list in list', 'payload': {'column1': 'Y', 'column2': [1, 2, []], 'column3': {'A': 1}}}

Sorry to forget that

Error:
TypeError                                 Traceback (most recent call last)
<ipython-input-61-8adcda79951d> in <module>
    112     payload2 = []
    113     for i in range(len(payload)):
--> 114         for value in i['payload'].value():
    115             if value =='':
    116                 break

TypeError: 'int' object is not subscriptable

I had this error

That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.

if isinstance(payload, list):
    payload = [d for d in payload if not "" in d["payload"].values()]
else:
    payload = {k:v for k, v in payload.items() if v != ""}

I still think a more generic solution is better.

(Jan-11-2022, 10:15 PM)deanhystad Wrote: [ -> ]That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.

if isinstance(payload, list):
    payload = [d for d in payload if not "" in d["payload"].values()]
else:
    payload = {k:v for k, v in payload.items() if v != ""}

I still think a more generic solution is better.

Thank You