Python Forum
Remove empty keys in a python list
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Remove empty keys in a python list
#1
I have a list of dictionaries and in case if some dictionary has empty key I want to remove it completely from the list. In my example,
I need to remove all third block because column3 is empty.

a = [
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 98
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 180
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": ""
        }
    }    
]
This is my code but is not removing and I think it's because of json structure

a = [d for d in payload if "" not in d.values()]

print(type(a))
<class 'list'>
Does anyone knows how can I solve it?
Reply
#2
I couldn't figure out how to do it with list comprehension but here's something that at least does the job.
a = [
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 98
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": 180
        }
    },
    {
        "type": "some.type2",
        "payload": {
            "column1": "12345",
            "column2": "description",
            "column3": ""
        }
    }    
]
z = [] 

for dictionary in a :
	for value in dictionary ['payload'].values () :
		if value == '' :
			break
	else :
		z.append (dictionary)

for dictionary in z :
	print (dictionary)
Reply
#3
I have tried this solution, but did not work. Here is the code I have tried. I have "if conditional" from type because I have others type. This specially is with problem

print(type(a))
payload = json.loads(json.dumps(a))

if type(payload) == list:
    payload2 = []
    for dictionary in payload:
        for value in dictionary['payload'].value():
            if value =='':
                break
            else:
                payload2.append(dictionary)
    payload = payload2
else:
    payload = {k:v for (k,v) in payload.items() if v != ""}


message['messages'] = payload
print(json.dumps( message ))
Any idea?
Reply
#4
Please elaborate "but did not work". Did you get an error message? Or an unexpected result? Or what.
Reply
#5
The tricky part is to "flatten" the dictionary so you look at all the keys.
a = [
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 0
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": None
        }
    },
    {
        "type": "X",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": {}
        }
    }
]

def empty_key(dictionary):
    """Recursively scan dictionary for values that are empty collections or None.
    Return associated key and value or None if no 'empty' values found."""
    for key in dictionary.keys():
        value = dictionary[key]
        if value is None:
            return key, value
        elif isinstance(value, (list, tuple, set, str)) and not value:
            return key, value
        elif isinstance(value, dict):
            if value:
                if (value := empty_key(value)):
                     return value
            else:
                return key, value  # Empty dictionary
    return None

print([x for x in a if empty_key(x) is None])
This is easily expanded to not only scan dictionary values but list, tuple, and set values as well.
a = [
    {
        "type": "Not empty",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "Zero, Not empty",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": 0
        }
    },
    {
        "type": "column1 is empty str",
        "payload": {
            "column1": "",
            "column2": [1, 2, 3],
            "column3": 1
        }
    },
    {
        "type": "column3 is None",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": None
        }
    },
    {
        "type": "column3 is empty dictionary",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, 3],
            "column3": {}
        }
    },
    {
        "type": "Column2 has empty list in list",
        "payload": {
            "column1": "Y",
            "column2": [1, 2, []],
            "column3": {"A":1}
        }
    }
]

def isempty(thing):
    """Recursively scan for emptyness.  Return True if emptyness is found
    in the form of an empty string, list, tuple, set or dictionary."""
    if isinstance(thing, str):
        if not thing:
            return True
    elif isinstance(thing, (list, tuple, set)):
        if not thing:
            return True
        for subthing in thing:
            if isempty(subthing):
                return True
    elif isinstance(thing, dict):
        if not thing:
            return True
        for subthing in thing.values():
            if isempty(subthing):
                return True
    return False

for thing in [x for x in a if isempty(x)]:
    print(thing)
Output:
{'type': 'column1 is empty str', 'payload': {'column1': '', 'column2': [1, 2, 3], 'column3': 1}} {'type': 'column3 is None', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': None}} {'type': 'column3 is empty dictionary', 'payload': {'column1': 'Y', 'column2': [1, 2, 3], 'column3': {}}} {'type': 'Column2 has empty list in list', 'payload': {'column1': 'Y', 'column2': [1, 2, []], 'column3': {'A': 1}}}
Reply
#6
Sorry to forget that

Error:
TypeError Traceback (most recent call last) <ipython-input-61-8adcda79951d> in <module> 112 payload2 = [] 113 for i in range(len(payload)): --> 114 for value in i['payload'].value(): 115 if value =='': 116 break TypeError: 'int' object is not subscriptable
I had this error
Reply
#7
That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.
if isinstance(payload, list):
    payload = [d for d in payload if not "" in d["payload"].values()]
else:
    payload = {k:v for k, v in payload.items() if v != ""}
I still think a more generic solution is better.
Reply
#8
Thumbs Up 
(Jan-11-2022, 10:15 PM)deanhystad Wrote: That error trace is not for the code you posted.

Your error is dictionary does not have a method named "value", it has "values".

I think you can do this as a one liner. Well two one liners.
if isinstance(payload, list):
    payload = [d for d in payload if not "" in d["payload"].values()]
else:
    payload = {k:v for k, v in payload.items() if v != ""}
I still think a more generic solution is better.

Thank You
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  unable to remove all elements from list based on a condition sg_python 3 373 Jan-27-2024, 04:03 PM
Last Post: deanhystad
  Code with empty list not executing adeana 9 3,635 Dec-11-2023, 08:27 AM
Last Post: buran
  Remove numbers from a list menator01 4 1,251 Nov-13-2022, 01:27 AM
Last Post: menator01
  set.difference of two list gives empty result wardancer84 4 1,433 Jun-14-2022, 01:36 PM
Last Post: wardancer84
  Updating nested dict list keys tbaror 2 1,243 Feb-09-2022, 09:37 AM
Last Post: tbaror
  displaying empty list vlearner 5 1,606 Jan-19-2022, 09:12 AM
Last Post: perfringo
  Remove an item from a list contained in another item in python CompleteNewb 19 5,547 Nov-11-2021, 06:43 AM
Last Post: Gribouillis
  Compile list of dictianories out of another list of dictianories by certain keys CatorCanulis 10 4,030 Jun-10-2021, 08:35 PM
Last Post: perfringo
  .remove() from a list - request for explanation InputOutput007 3 2,177 Jan-28-2021, 04:21 PM
Last Post: InputOutput007
  What is the value after JOINING an empty list? JaneTan 2 5,061 Jan-04-2021, 06:25 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020