Python Forum
alternative to nested loops for large data set
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
alternative to nested loops for large data set
#1
Hi, I've created some terribly slow code and I'd like to learn how I can improve it.

I want to turn the ownedcards dictionary of card id's and quantities into a list of card names and quantities.

ownedcards = {16: 2, 20: 139}

card_sections = [{'card_id': '16',
  'card_set': '1000',
  'fully_upgraded': '',
  'fusion_level': '0',
  'level': '1',
  'name': 'Terminator-1',
  'rarity': '2',
  'sp_value': '5'},
 {'card_id': '20',
  'card_set': '1000',
  'fully_upgraded': '',
  'fusion_level': '',
  'level': '3',
  'name': 'Benediction-3',
  'rarity': '4',
  'sp_value': '60'}]


card_names = []
for (card_id, qty) in ownedcards.items():
    for key in card_sections:
        if card_id == key['card_id']:
            card_names.append(key['name'] + ' #' + str(qty))


card_names outputs:

 ['Terminator-1 #2',
 'Benediction-3 #139']
card_sections is a list of 46750 dicts and the ownedcards dict can be 1000+ elements long so the above solution is really slow.

any advice would be appreciated
Reply
#2
You could start by grouping the card sections by their card_id. There are several ways to do that, here is one
from operator import itemgetter
import itertools

dic = {int(key): list(group) for key, group in itertools.groupby(
            sorted(card_sections, key=itemgetter('card_id')), key=itemgetter('card_id')))}

card_names = []
for card_id, qty in owned_cards.items():
    for sections in dic.get(card_id, []):
        card_names.extend('{}#{}'.format(s['name'], qty) for s in sections)
Alternately you could use more_itertools.bucket() if you have this module. Also note that in Python, 20 != '20'
Reply
#3
Great, thanks. Thats an interesting solution, I'll have to spend some time trying wrap my head around it.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Break out of nested loops muzikman 11 992 Sep-18-2021, 12:59 PM
Last Post: muzikman
  How to break out of nested loops pace 11 2,154 Mar-03-2021, 06:25 PM
Last Post: pace
  Nested for Loops sammay 1 794 Jan-09-2021, 06:48 PM
Last Post: deanhystad
  Twilio alternative jmair 3 2,148 Jan-04-2021, 10:22 AM
Last Post: richardonline
  Data Loops ZairaRosas 0 674 Dec-21-2020, 11:07 PM
Last Post: ZairaRosas
  Alternative for Cairosvg? Maryan 0 957 Oct-26-2020, 01:27 PM
Last Post: Maryan
  Looping Through Large Data Sets JoeDainton123 10 1,795 Oct-18-2020, 02:58 PM
Last Post: buran
  Extract data from large string pzig98 1 1,046 Jul-20-2020, 12:39 AM
Last Post: Larz60+
  How to make this function general to create binary numbers? (many nested for loops) dospina 4 1,528 Jun-24-2020, 04:05 AM
Last Post: deanhystad
  Python beginner - nested while loops mikebarden 1 895 Jun-01-2020, 01:04 PM
Last Post: DPaul

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020