Python Forum
alternative to nested loops for large data set
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
alternative to nested loops for large data set
#1
Hi, I've created some terribly slow code and I'd like to learn how I can improve it.

I want to turn the ownedcards dictionary of card id's and quantities into a list of card names and quantities.

ownedcards = {16: 2, 20: 139}

card_sections = [{'card_id': '16',
  'card_set': '1000',
  'fully_upgraded': '',
  'fusion_level': '0',
  'level': '1',
  'name': 'Terminator-1',
  'rarity': '2',
  'sp_value': '5'},
 {'card_id': '20',
  'card_set': '1000',
  'fully_upgraded': '',
  'fusion_level': '',
  'level': '3',
  'name': 'Benediction-3',
  'rarity': '4',
  'sp_value': '60'}]


card_names = []
for (card_id, qty) in ownedcards.items():
    for key in card_sections:
        if card_id == key['card_id']:
            card_names.append(key['name'] + ' #' + str(qty))


card_names outputs:

 ['Terminator-1 #2',
 'Benediction-3 #139']
card_sections is a list of 46750 dicts and the ownedcards dict can be 1000+ elements long so the above solution is really slow.

any advice would be appreciated
Reply
#2
You could start by grouping the card sections by their card_id. There are several ways to do that, here is one
from operator import itemgetter
import itertools

dic = {int(key): list(group) for key, group in itertools.groupby(
            sorted(card_sections, key=itemgetter('card_id')), key=itemgetter('card_id')))}

card_names = []
for card_id, qty in owned_cards.items():
    for sections in dic.get(card_id, []):
        card_names.extend('{}#{}'.format(s['name'], qty) for s in sections)
Alternately you could use more_itertools.bucket() if you have this module. Also note that in Python, 20 != '20'
Reply
#3
Great, thanks. Thats an interesting solution, I'll have to spend some time trying wrap my head around it.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  breaking out of nested loops Skaperen 3 244 Jul-18-2022, 12:59 AM
Last Post: Skaperen
  Convert nested sample json api data into csv in python shantanu97 3 432 May-21-2022, 01:30 PM
Last Post: deanhystad
  Break out of nested loops muzikman 11 1,627 Sep-18-2021, 12:59 PM
Last Post: muzikman
  How to break out of nested loops pace 11 3,009 Mar-03-2021, 06:25 PM
Last Post: pace
  Nested for Loops sammay 1 1,180 Jan-09-2021, 06:48 PM
Last Post: deanhystad
  Twilio alternative jmair 3 2,543 Jan-04-2021, 10:22 AM
Last Post: richardonline
  Data Loops ZairaRosas 0 909 Dec-21-2020, 11:07 PM
Last Post: ZairaRosas
  Alternative for Cairosvg? Maryan 0 1,340 Oct-26-2020, 01:27 PM
Last Post: Maryan
  Looping Through Large Data Sets JoeDainton123 10 2,514 Oct-18-2020, 02:58 PM
Last Post: buran
  Extract data from large string pzig98 1 1,398 Jul-20-2020, 12:39 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020