Python Forum
Recommendation for data structure
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Recommendation for data structure
#1
Hi,
I'm self-teaching myself Python using examples and the excellent documentation.
My project is a program that uses a single random seed to generate a made-up "record label", with multiple "artists", each with multiple "albums", each with multiple "tracks", which are generated CSound music code files. It also names the artists, albums, tracks and members of each band, and it generates artwork and album covers.
(If anyone was interested in learning more about this project, PM me. I'd be glad of collaborators!)

I am currently storing the generated data in dictionaries, for example:
label_data = {label_name: "xxx", num_artists: 3, etc etc}
artist_data = {artist_name: "yyy", artist_id: (sequential), num_albums: 5, etc etc}
album_data = {id, name, year, num_tracks, etc etc}
track_data = {id, bpm, tune_pattern, snare_drum_pattern, etc etc}

My question is, would nesting these structures be efficient? The number of objects would vary, due to the randomness.
1 label
1-10 artists
each with 1-10 albums
each with 1-10 tracks
This would mean quite a large number of objects, with 4 levels of nesting.

Would a different data structure be better, for example SQLite? Then I could relate the various objects together using ids and queries.

Grateful for any suggestions.

Best regards,
Matthew Petty
Dubai
Reply
#2
I like to nest dictionaries of related material.
It's easy to do, and then I save as json.
see CreateFIPScodeDescJson.py  listing here: https://python-forum.io/Thread-Practical-use-of-pathlib
Reply
#3
You mix what structure will be used for in-memory manipulation of data and what to use to store data on disk.
sqlite or any other database to that matter will make difference how you retrieve information from disk storage, but you can put this information in whatever data structure you want to manipulate it in memory. e.g. in your other post you were asking about classes, but now I see you still not use classes. As larz60+ suggest you may use also json to store the info on disk, but if this is going to be a serious long-term project I would opt for sqlite. It will be relatively easy to move to different [relational] DB later if you decide so.
Nested dictionaries or not will make difference on how you will retrieve individual records. You can have nested dictionaries also with relational database as backend.
Reply
#4
(Oct-30-2017, 07:17 AM)buran Wrote: You mix what structure will be used for in-memory manipulation of data and what to use to store data on disk.
sqlite or any other database to that matter will make difference how you retrieve information from disk storage, but you can put this information in whatever data structure you want to manipulate it in memory. e.g. in your other post you were asking about classes, but now I see you still not use classes. As larz60+ suggest you may use also json to store the info on disk, but if this is going to be a serious long-term project I would opt for sqlite. It will be relatively easy to move to different [relational] DB later if you decide so.
Nested dictionaries or not will make difference on how you will retrieve individual records. You can have nested dictionaries also with relational database as backend.

Hi buran,
The data is only used once, to generate other files, although it would be good to dump the data used in a file for reference. I think I will stick with dictionaries for now.

I have not been able to figure out how to change my code to use classes. I've replied to that thread.

Thanks,
Matthew
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How can I add certain elements in this 2d data structure and calculate a mean TheOddCircle 3 1,508 May-27-2022, 09:09 AM
Last Post: paul18fr
  CPU recommendation for Python (11900K or 5950X) MangirdTea 2 1,848 Sep-08-2021, 11:00 PM
Last Post: Larz60+
  Appropriate data-structure / design for business-day relations (week/month-wise) sx999 2 2,750 Apr-23-2021, 08:09 AM
Last Post: sx999
  what data structure to use? Winfried 4 2,784 Mar-16-2021, 12:11 PM
Last Post: buran
  Yahoo_fin, Pandas: how to convert data table structure in csv file detlefschmitt 14 7,559 Feb-15-2021, 12:58 PM
Last Post: detlefschmitt
  How to use Bunch data structure moish 2 2,832 Dec-24-2020, 06:25 PM
Last Post: deanhystad
  Correct data structure for this problem Wigi 13 4,503 Oct-09-2020, 11:09 AM
Last Post: buran
  difficulties to chage json data structure using json module in python Sibdar 1 2,043 Apr-03-2020, 06:47 PM
Last Post: micseydel
  File system representation in a data structure Alfalfa 1 2,027 Dec-18-2019, 01:56 AM
Last Post: Alfalfa
  [ library recommendation ] search & highlight target texts in PDF? smallabc 1 2,193 Nov-27-2019, 10:40 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020