Python Forum
Finding value in nested dictionaries with lists
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Finding value in nested dictionaries with lists
#1
I am struggling with finding a value within a dictionary containing nested dictionaries which in turn can contain nested dictionaries, containing lists etc. The value, to be found, can be anywhere, so within a nested dictionary in a nested dictionary in a list, but also within a list within a nested dictionary or just a list. If the value is found more than once, I also want to know the name of the dictionary to which it belongs.

For example: If the value is found in a list and that list belongs to a nested dictionary, I want to know the name of that nested dictionary.

Unfortunately, I am not able to give you an example. Can you give me some pointers who I could solve this problem?

I have thought about it:
Would flatten the dictionary be an option and look trough the values to see if they match and then return their respective keys? The problem is that I do not know how to flatten complex nested dictionaries....
Reply
#2
At least post the dictionary (or sample of if large) showing the nesting.
access should be quite simple
>>> outerdict = {
...     'value1': 45,
...     'innerdict': {
...         'value2': 75,
...         'value3': 'Peter'
...     }
... }
>>> print(outerdict['innerdict']['value3'])
Peter
>>>
Reply
#3
(Mar-07-2020, 09:18 PM)Larz60+ Wrote: At least post the dictionary (or sample of if large) showing the nesting.
access should be quite simple
>>> outerdict = {
...     'value1': 45,
...     'innerdict': {
...         'value2': 75,
...         'value3': 'Peter'
...     }
... }
>>> print(outerdict['innerdict']['value3'])
Peter
>>>

I understand, but I do not know the key of the value, so I can't reference to it directly. I only know the value, and need to return the key to know if the value is found at the correct entry. This due to possible duplicates in the dictionary.

dict = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
The above is an example. I am looking for the value 'ABCD' and need to return 'nestedA'+'VarE', 'nestedB'+'LocA'+'v1' and 'nestedB'+'LocA'+'v4'+'name'. This because it is presented three times.
Reply
#4
you can find all keys at any level
ex
keys = outerdict.keys()
you can access key and value with for key, value in outerdict.items()
Reply
#5
isinstance() is the key to the solution of your problem.
You should make a function that starts with identifying the type.
If the passed parameter is an instance of "dict" then iterate over the values. (Indeed as Larz60+ wrote: for key, value in outerdict.items().) For each value call the same function recursively.
If the passed parameter is an instance of "list" then iterate over the values. For each value call the same function recursively.
If the passed parameter is an instance of "str" then compare the value with the value you are looking for. If it is the same: BINGO.

The first parameter of the function should be the object (dictionary, list, ...) to be inspected.
The second parameter of the function should be the value to look for.
The third parameter must default to a null string and should contain the indexpath. For each iteration the indexpath has to be extended. You use the indexpath as an answer to where the value was found.
In the simplest form the function may print the indexpath when a match is found.
But a well formed function usually returns the answer. This then would be a list of values because you state the value may occur more than once. This list should also be passed as a parameter and returned as an answer. For each match found, the indexpath must be added to this list.
I hope this helps you.
Reply
#6
(Mar-08-2020, 10:57 AM)ibreeden Wrote: isinstance() is the key to the solution of your problem.
You should make a function that starts with identifying the type.
If the passed parameter is an instance of "dict" then iterate over the values. (Indeed as Larz60+ wrote: for key, value in outerdict.items().) For each value call the same function recursively.
If the passed parameter is an instance of "list" then iterate over the values. For each value call the same function recursively.
If the passed parameter is an instance of "str" then compare the value with the value you are looking for. If it is the same: BINGO.

The first parameter of the function should be the object (dictionary, list, ...) to be inspected.
The second parameter of the function should be the value to look for.
The third parameter must default to a null string and should contain the indexpath. For each iteration the indexpath has to be extended. You use the indexpath as an answer to where the value was found.
In the simplest form the function may print the indexpath when a match is found.
But a well formed function usually returns the answer. This then would be a list of values because you state the value may occur more than once. This list should also be passed as a parameter and returned as an answer. For each match found, the indexpath must be added to this list.
I hope this helps you.

I understand it partially but get stuck at recursively...
    for key, value in dictionary.items():
        if isinstance(value, (dict)):
            for key1, value1 in value.items():
                if isinstance(value1, list):
                    for index in value1:
                        print(key, key1, index)
Reply
#7
Hello Mart79,
No, that is not what I mean. I meant this:
from collections import OrderedDict # the data appears to contain an OrderedDict.

def recurse_object(obj_to_inspect, val_to_find, indexpath=""):
    if isinstance(obj_to_inspect, dict):
        for key, value in obj_to_inspect.items():
            recurse_object(value, val_to_find, indexpath + f"['{key}']")
    if isinstance(obj_to_inspect, list):
        for key, value in enumerate(obj_to_inspect):
            recurse_object(value, val_to_find, indexpath + f"[{key}]")
    if isinstance(obj_to_inspect, str):
        if obj_to_inspect == val_to_find:
            print(f"Value {val_to_find} found at {indexpath}")

dictionary = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
recurse_object(dictionary, 'ABCD') 
Output:
Value ABCD found at ['nestedA']['VarE'] Value ABCD found at ['nestedB']['LocA']['v1'] Value ABCD found at ['nestedB']['LocA']['v4'][2][1]
By the way: You used dict = {'nestedA': ... etc. Never use reserved words (like dict) as a name of a variable.
Reply
#8
(Mar-08-2020, 01:31 PM)ibreeden Wrote: Hello Mart79,
No, that is not what I mean. I meant this:
from collections import OrderedDict # the data appears to contain an OrderedDict.

def recurse_object(obj_to_inspect, val_to_find, indexpath=""):
    if isinstance(obj_to_inspect, dict):
        for key, value in obj_to_inspect.items():
            recurse_object(value, val_to_find, indexpath + f"['{key}']")
    if isinstance(obj_to_inspect, list):
        for key, value in enumerate(obj_to_inspect):
            recurse_object(value, val_to_find, indexpath + f"[{key}]")
    if isinstance(obj_to_inspect, str):
        if obj_to_inspect == val_to_find:
            print(f"Value {val_to_find} found at {indexpath}")

dictionary = {'nestedA': OrderedDict([('Name', 'TestCase'), ('VarA', 'Local'), ('VarB', None), ('VarC', None), ('VarD', [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]), ('VarE', 'ABCD'), ('VarF', 10.0), ('VarG', False)]), 'nestedB': OrderedDict([('LocA', {'v1': 'ABCD', 'v2': None, 'v3': None, 'v4': [[[0, 1], [1, 1], [2, 2], [0, 4]], [[0, 2], [1, 2], [2, 3], [4, 4]], ['name', 'ABCD']]}), ('LocB', None), ('LocC', None), ('LocD', True), ('LocE', True), ('LocF', False), ('LocG', False)])}
recurse_object(dictionary, 'ABCD') 
Output:
Value ABCD found at ['nestedA']['VarE'] Value ABCD found at ['nestedB']['LocA']['v1'] Value ABCD found at ['nestedB']['LocA']['v4'][2][1]
By the way: You used dict = {'nestedA': ... etc. Never use reserved words (like dict) as a name of a variable.

@ ibreeden, thank you so much! Yes, I did not realized it at first that I was using a reserved word.
Unfortunately, I have a second problem. The dictionary contains object information from other objects which do not get searched through.
I can't explain it other than an example:

dictionary = <class 'dict'>: {'case': 'Test',......,'checker': <checking.check.assessment object at 0x0003432C2D5646EB8>
the checking.check.assessment contains object information, dictionaries, lists etc. too.

How do I search through these objects for the same 'ABCD'?
Reply
#9
I do not understand what sort of object information you mean. It seems like it contains XML? Or is it something else?
You can add if statements to handle those elements.
Reply
#10
(Mar-08-2020, 03:55 PM)ibreeden Wrote: I do not understand what sort of object information you mean. It seems like it contains XML? Or is it something else?
You can add if statements to handle those elements.

This referenced object contains dictionaries and lists also, but for some reason the code does not search through these dictionaries and lists.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  List all possibilities of a nested-list by flattened lists sparkt 1 878 Feb-23-2023, 02:21 PM
Last Post: sparkt
  Split dict of lists into smaller dicts of lists. pcs3rd 3 2,312 Sep-19-2020, 09:12 AM
Last Post: ibreeden
  Comparing items from 2 lists of dictionaries illwill 7 2,661 Sep-14-2020, 10:46 PM
Last Post: bowlofred
  Searching through Nested Dictionaries and Lists Dave_London 1 6,240 Jul-09-2020, 03:36 PM
Last Post: mrdominikku
  Creating Nested Dictionaries Confusion gw1500se 2 2,079 May-18-2020, 11:16 PM
Last Post: gw1500se
  Unpacking nested lists yonatan776 1 2,158 Apr-14-2020, 08:50 PM
Last Post: buran
  Help: for loop with dictionary and nested lists mart79 1 1,833 Apr-12-2020, 02:52 PM
Last Post: TomToad
  nested dictionaries to CSV mart79 9 12,299 Jul-29-2019, 04:59 AM
Last Post: mart79
  Transform simplified dictionary to nested dictionaries bhojendra 1 2,323 Jul-02-2019, 02:05 PM
Last Post: ichabod801
  sort lists of lists with multiple criteria: similar values need to be treated equal stillsen 2 3,187 Mar-20-2019, 08:01 PM
Last Post: stillsen

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020