+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: alternative to nested loops for large data set (/thread-24564.html)
alternative to nested loops for large data set - JonnyEnglish - Feb-19-2020
Hi, I've created some terribly slow code and I'd like to learn how I can improve it.
I want to turn the ownedcards dictionary of card id's and quantities into a list of card names and quantities.
ownedcards = {16: 2, 20: 139}
card_sections = [{'card_id': '16',
'card_set': '1000',
'fully_upgraded': '',
'fusion_level': '0',
'level': '1',
'name': 'Terminator-1',
'rarity': '2',
'sp_value': '5'},
{'card_id': '20',
'card_set': '1000',
'fully_upgraded': '',
'fusion_level': '',
'level': '3',
'name': 'Benediction-3',
'rarity': '4',
'sp_value': '60'}]
card_names = []
for (card_id, qty) in ownedcards.items():
for key in card_sections:
if card_id == key['card_id']:
card_names.append(key['name'] + ' #' + str(qty))
card_names outputs:
['Terminator-1 #2',
'Benediction-3 #139']card_sections is a list of 46750 dicts and the ownedcards dict can be 1000+ elements long so the above solution is really slow. any advice would be appreciated
RE: alternative to nested loops for large data set - Gribouillis - Feb-19-2020
You could start by grouping the card sections by their card_id. There are several ways to do that, here is one
from operator import itemgetter
import itertools
dic = {int(key): list(group) for key, group in itertools.groupby(
sorted(card_sections, key=itemgetter('card_id')), key=itemgetter('card_id')))}
card_names = []
for card_id, qty in owned_cards.items():
for sections in dic.get(card_id, []):
card_names.extend('{}#{}'.format(s['name'], qty) for s in sections)Alternately you could use more_itertools.bucket() if you have this module. Also note that in Python, 20 != '20'
RE: alternative to nested loops for large data set - JonnyEnglish - Feb-19-2020
Great, thanks. Thats an interesting solution, I'll have to spend some time trying wrap my head around it.