Suggestions: Datastructure Design

[wisdomofnimn]

I've been using python for 3-4 years (pseudo-)professionally, but one topic that I struggle with routinely is what I'd call Data Structures but I feel has a better name. I'm referring not to linked lists, binary trees and the like, but rather to the composition of data into (most often) lists, dictionaries, etc and passed through the script to achieve its function. I hope I'm being clear - I am certain this not the industry's term because searches are not turning up what I'm looking for.

I mostly write automation for networks and business OSS/BSS - and one huge time sink is creating a structure to hold pertinent details for network devices, orders, services, whatever i'm automating, then realize I need to reorganize it which breaks previously working code until I go through all the classes/functions to update. I know there has to be a better way.

I am looking for suggestions in terms of books, video series, anecdotes or advice for designing these structures in general.

Thanks

[Larz60+]

I don't know if this is of interest to you or not.
I wrote this several years ago, and use it all the time for dynamic dictionary creation.
It's simple to use.
Code includes a dynamic sample.
Use it if you can and wish to do so.

CreateDict.py

import os

class CreateDict:
    """
    Generic Dynamic Dictionary tool.

    CreateDict.py - Contains methods to simplify node and cell creation within
                    a dictionary

    Usage: 
    
        The best way to learn what can be done is to examine the testit function
        included in this module.

        new_dict(dictname) - Creates a new dictionary instance with the name
            contained in dictname

        add_node(parent, nodename) - Creates a new node (nested dictionary)
            named in nodename, in parent dictionary.

        add_cell(nodename, cellname, value) - Creates a leaf node within node
            named in nodename, with a cell name of cellname, and value of value.

        display_dict(dictname) - Recursively displays a nested dictionary.

    Requirements:

        Python standard library:
            os
    
    Author: Larz60+  -- May 2019.
    """
    def __init__(self):
        os.chdir(os.path.abspath(os.path.dirname(__file__)))

    def new_dict(self, dictname):
        setattr(self, dictname, {})

    def add_node(self, parent, nodename):
        node = parent[nodename] = {}
        return node

    def add_cell(self, nodename, cellname, value):
        cell =  nodename[cellname] = value
        return cell

    def display_dict(self, dictname, level=0):
        indent = " " * (4 * level)
        for key, value in dictname.items():
            if isinstance(value, dict):
                print(f'\n{indent}{key}')
                level += 1
                self.display_dict(value, level)
            else:
                print(f'{indent}{key}: {value}')
            if level > 0:
                level -= 1


def testit():
    # instantiate class
    cd = CreateDict()

    # create new dictionary named CityList
    cd.new_dict('CityList')

    # add node Boston
    boston = cd.add_node(cd.CityList, 'Boston')
    # add sub node Resturants
    bos_resturants = cd.add_node(boston, 'Resturants')

    # Add subnode 'Spoke Wine Bar' to parent bos_resturants
    spoke = cd.add_node(bos_resturants, 'Spoke Wine Bar')
    cd.add_cell(spoke, 'Addr1', '89 Holland St')
    cd.add_cell(spoke, 'City', 'Sommerville')
    cd.add_cell(spoke, 'Addr1', '02144')
    cd.add_cell(spoke, 'Phone', '617-718-9463')

    # Add subnode 'Highland Kitchen' to parent bos_resturants
    highland = cd.add_node(bos_resturants, 'Highland Kitchen')
    cd.add_cell(highland, 'Addr1', '150 Highland Ave')
    cd.add_cell(highland, 'City', 'Sommerville')
    cd.add_cell(highland, 'ZipCode', '02144')
    cd.add_cell(highland, 'Phone', '617-625-1131')

    # display dictionary
    print(f'\nCityList Dictionary')
    cd.display_dict(cd.CityList)
    print(f'\nraw data: {cd.CityList}')

if __name__ == '__main__':
    testit()

output of test routine:

Output:
CityList Dictionary

Boston

    Resturants

        Spoke Wine Bar
            Addr1: 02144
            City: Sommerville
            Phone: 617-718-9463

        Highland Kitchen
            Addr1: 150 Highland Ave
            City: Sommerville
            ZipCode: 02144
            Phone: 617-625-1131

raw data: {'Boston': {'Resturants': {'Spoke Wine Bar': {'Addr1': '02144', 'City': 'Sommerville', 'Phone': '617-718-9463'}, 'Highland Kitchen': {'Addr1': '150 Highland Ave', 'City': 'Sommerville', 'ZipCode': '02144', 'Phone': '617-625-1131'}}}}

[ndc85430]

I'm not too sure what you're asking really. Is the main problem that you're finding you have to change code a lot? That is what happens with software, as the needs of its users change. Do you have automated tests in place, so that you can refactor your code and still have confidence that it works? If not, that is definitely something you'll want to do.