Python Forum
Finding value in nested dictionaries with lists
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Finding value in nested dictionaries with lists
#1
I am struggling with finding a value within a dictionary containing nested dictionaries which in turn can contain nested dictionaries, containing lists etc. The value, to be found, can be anywhere, so within a nested dictionary in a nested dictionary in a list, but also within a list within a nested dictionary or just a list. If the value is found more than once, I also want to know the name of the dictionary to which it belongs.

For example: If the value is found in a list and that list belongs to a nested dictionary, I want to know the name of that nested dictionary.

Unfortunately, I am not able to give you an example. Can you give me some pointers who I could solve this problem?

I have thought about it:
Would flatten the dictionary be an option and look trough the values to see if they match and then return their respective keys? The problem is that I do not know how to flatten complex nested dictionaries....
Reply
#2
At least post the dictionary (or sample of if large) showing the nesting.
access should be quite simple
>>> outerdict = {
...     'value1': 45,
...     'innerdict': {
...         'value2': 75,
...         'value3': 'Peter'
...     }
... }
>>> print(outerdict['innerdict']['value3'])
Peter
>>>
Reply
#3
(Mar-07-2020, 09:18 PM)Larz60+ Wrote: At least post the dictionary (or sample of if large) showing the nesting.
access should be quite simple
>>> outerdict = {
...     'value1': 45,
...     'innerdict': {
...         'value2': 75,
...         'value3': 'Peter'
...     }
... }
>>> print(outerdict['innerdict']['value3'])
Peter
>>>

I understand, but I do not know the key of the value, so I can't reference to it directly. I only know the value, and need to return the key to know if the value is found at the correct entry. This due to possible duplicates in the dictionary.

dict = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
The above is an example. I am looking for the value 'ABCD' and need to return 'nestedA'+'VarE', 'nestedB'+'LocA'+'v1' and 'nestedB'+'LocA'+'v4'+'name'. This because it is presented three times.
Reply
#4
you can find all keys at any level
ex
keys = outerdict.keys()
you can access key and value with for key, value in outerdict.items()
Reply
#5
isinstance() is the key to the solution of your problem.
You should make a function that starts with identifying the type.
If the passed parameter is an instance of "dict" then iterate over the values. (Indeed as Larz60+ wrote: for key, value in outerdict.items().) For each value call the same function recursively.
If the passed parameter is an instance of "list" then iterate over the values. For each value call the same function recursively.
If the passed parameter is an instance of "str" then compare the value with the value you are looking for. If it is the same: BINGO.

The first parameter of the function should be the object (dictionary, list, ...) to be inspected.
The second parameter of the function should be the value to look for.
The third parameter must default to a null string and should contain the indexpath. For each iteration the indexpath has to be extended. You use the indexpath as an answer to where the value was found.
In the simplest form the function may print the indexpath when a match is found.
But a well formed function usually returns the answer. This then would be a list of values because you state the value may occur more than once. This list should also be passed as a parameter and returned as an answer. For each match found, the indexpath must be added to this list.
I hope this helps you.
Reply
#6
(Mar-08-2020, 10:57 AM)ibreeden Wrote: isinstance() is the key to the solution of your problem.
You should make a function that starts with identifying the type.
If the passed parameter is an instance of "dict" then iterate over the values. (Indeed as Larz60+ wrote: for key, value in outerdict.items().) For each value call the same function recursively.
If the passed parameter is an instance of "list" then iterate over the values. For each value call the same function recursively.
If the passed parameter is an instance of "str" then compare the value with the value you are looking for. If it is the same: BINGO.

The first parameter of the function should be the object (dictionary, list, ...) to be inspected.
The second parameter of the function should be the value to look for.
The third parameter must default to a null string and should contain the indexpath. For each iteration the indexpath has to be extended. You use the indexpath as an answer to where the value was found.
In the simplest form the function may print the indexpath when a match is found.
But a well formed function usually returns the answer. This then would be a list of values because you state the value may occur more than once. This list should also be passed as a parameter and returned as an answer. For each match found, the indexpath must be added to this list.
I hope this helps you.

I understand it partially but get stuck at recursively...
    for key, value in dictionary.items():
        if isinstance(value, (dict)):
            for key1, value1 in value.items():
                if isinstance(value1, list):
                    for index in value1:
                        print(key, key1, index)
Reply
#7
Hello Mart79,
No, that is not what I mean. I meant this:
from collections import OrderedDict # the data appears to contain an OrderedDict.

def recurse_object(obj_to_inspect, val_to_find, indexpath=""):
    if isinstance(obj_to_inspect, dict):
        for key, value in obj_to_inspect.items():
            recurse_object(value, val_to_find, indexpath + f"['{key}']")
    if isinstance(obj_to_inspect, list):
        for key, value in enumerate(obj_to_inspect):
            recurse_object(value, val_to_find, indexpath + f"[{key}]")
    if isinstance(obj_to_inspect, str):
        if obj_to_inspect == val_to_find:
            print(f"Value {val_to_find} found at {indexpath}")

dictionary = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
recurse_object(dictionary, 'ABCD') 
Output:
Value ABCD found at ['nestedA']['VarE'] Value ABCD found at ['nestedB']['LocA']['v1'] Value ABCD found at ['nestedB']['LocA']['v4'][2][1]
By the way: You used dict = {'nestedA': ... etc. Never use reserved words (like dict) as a name of a variable.
Reply
#8
(Mar-08-2020, 01:31 PM)ibreeden Wrote: Hello Mart79,
No, that is not what I mean. I meant this:
from collections import OrderedDict # the data appears to contain an OrderedDict.

def recurse_object(obj_to_inspect, val_to_find, indexpath=""):
    if isinstance(obj_to_inspect, dict):
        for key, value in obj_to_inspect.items():
            recurse_object(value, val_to_find, indexpath + f"['{key}']")
    if isinstance(obj_to_inspect, list):
        for key, value in enumerate(obj_to_inspect):
            recurse_object(value, val_to_find, indexpath + f"[{key}]")
    if isinstance(obj_to_inspect, str):
        if obj_to_inspect == val_to_find:
            print(f"Value {val_to_find} found at {indexpath}")

dictionary = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
recurse_object(dictionary, 'ABCD') 
Output:
Value ABCD found at ['nestedA']['VarE'] Value ABCD found at ['nestedB']['LocA']['v1'] Value ABCD found at ['nestedB']['LocA']['v4'][2][1]
By the way: You used dict = {'nestedA': ... etc. Never use reserved words (like dict) as a name of a variable.

@ ibreeden, thank you so much! Yes, I did not realized it at first that I was using a reserved word.
Unfortunately, I have a second problem. The dictionary contains object information from other objects which do not get searched through.
I can't explain it other than an example:

dictionary = <class 'dict'>: {'case': 'Test',......,'checker': <checking.check.assessment object at 0x0003432C2D5646EB8>
the checking.check.assessment contains object information, dictionaries, lists etc. too.

How do I search through these objects for the same 'ABCD'?
Reply
#9
I do not understand what sort of object information you mean. It seems like it contains XML? Or is it something else?
You can add if statements to handle those elements.
Reply
#10
(Mar-08-2020, 03:55 PM)ibreeden Wrote: I do not understand what sort of object information you mean. It seems like it contains XML? Or is it something else?
You can add if statements to handle those elements.

This referenced object contains dictionaries and lists also, but for some reason the code does not search through these dictionaries and lists.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Comparing items from 2 lists of dictionaries illwill 7 560 Sep-14-2020, 10:46 PM
Last Post: bowlofred
  Searching through Nested Dictionaries and Lists Dave_London 1 544 Jul-09-2020, 03:36 PM
Last Post: mrdominikku
  Creating Nested Dictionaries Confusion gw1500se 2 662 May-18-2020, 11:16 PM
Last Post: gw1500se
  Unpacking nested lists yonatan776 1 651 Apr-14-2020, 08:50 PM
Last Post: buran
  Help: for loop with dictionary and nested lists mart79 1 711 Apr-12-2020, 02:52 PM
Last Post: TomToad
  nested dictionaries to CSV mart79 9 5,213 Jul-29-2019, 04:59 AM
Last Post: mart79
  Transform simplified dictionary to nested dictionaries bhojendra 1 853 Jul-02-2019, 02:05 PM
Last Post: ichabod801
  Help to flatten list of nested dictionaries shawbapmp 4 2,730 Feb-25-2019, 10:18 PM
Last Post: shawbapmp
  Nested Dictionaries with Lists BriHaug 1 1,108 Feb-23-2019, 02:10 AM
Last Post: ichabod801
  Help with Dictionaries and List of Lists artblinked 7 1,194 Feb-14-2019, 05:07 PM
Last Post: artblinked

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020