Python Forum
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Go around memory error
#1
I am coding a program which stores 12 numbers in a number of lists (each list has 12 numbers) in a new list. That list also contains a string. I want to do arithmetic operations with every possible combination from the lists (with one number from each list).

When I try with combinations from 5 lists, it works out great. But with 6 lists and more, I run into "memoryerror" if I use the Python 32 bit version. It seems like it never stops thinking if I use the 64 bit version. I have tried to use less decimals in the numbers, then it works better. But I would like to have some decimals there. The ultimate goal would actually be to use up to 15 numbers with up to 4 decimals in up to 8 lists! But 12 numbers in 8 lists would also be nice.

I use Python 3.8.0 [MSC v.1916 64 bit (AMD64)] on win32. My OS is Windows 10 64-bit.

I did some research and found people recommending other people with a similar problem to either use chunking, use databases or use a loop instead of a list. I have also thought about another solution which would be writing the program in C, but I don't know if that would work (I can't do it yet).

The question I have for you is: Given some examples of solutions above, what kind of approach would you recommend or do yourself if you'd run into the same problem? It does not have to be any of the above.
I am just asking for a general approach, not that you fix the problem for me.

I will provide you with the code that works (5 lists) and the code that does not work (6 lists).

Thank you so much!

p1 = (0.1, 0.17, 0.11, 0.12, 0.08, 0.01, 0.03, 0.04, 0.1, 0.12, 0.14, 0.05)
p2 = (0.11, 0.22, 0.12, 0.04, 0.28, 0.2, 0.34, 0.15, 0.16, 0.08, 0.02, 0.02)
p3 = (0.3, 0.1, 0.13, 0.1, 0.08, 0.02, 0.04, 0.05, 0.1, 0.12, 0.17, 0.04)
p4 = (0.2, 0.16, 0.12, 0.15, 0.09, 0.01, 0.05, 0.05, 0.1, 0.11, 0.17, 0.05)
p5 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)

s1 = (0.43, 0.13, 0.03, 0.02, 0.01, 0.04, 0.04, 0.04, 0.05, 0.04, 0.03, 0.01)
s2 = (0.3, 0.22, 0.15, 0.01, 0.02, 0.08, 0.12, 0.10, 0.03, 0.02, 0.01, 0.04)
s3 = (0.26, 0.22, 0.22, 0.13, 0.15, 0.016, 0.12, 0.18, 0.3, 0.22, 0.15, 0.01)
s4 = (0.47, 0.1, 0.03, 0.01, 0.02, 0.05, 0.09, 0.04, 0.02, 0.03, 0.03, 0.01)
s5 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)

n1 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n2 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n3 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n4 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n5 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")

a = 0
b = 0
c = 0
d = 0
e = 0

lista = []

while a < len(n1):
 while b < len(n2):
  while c < len(n3):
   while d < len(n4):
    while e < len(n5):
     lista.append([((p1[a] * p2[b] * p3[c] * p4[d] * p5[e]) / (s1[a] * s2[b] * s3[c] * s4[d] * s5[e])), n1[a] + "  " + n2[b] + "  " + n3[c] + "  " + n4[d] + "  " + n5[e]])
     e += 1
    e = 0
    d += 1
   d = 0
   c += 1
  c = 0
  b += 1
 b = 0
 a += 1
p1 = (0.1, 0.17, 0.11, 0.12, 0.08, 0.01, 0.03, 0.04, 0.1, 0.12, 0.14, 0.05)
p2 = (0.11, 0.22, 0.12, 0.04, 0.28, 0.2, 0.34, 0.15, 0.16, 0.08, 0.02, 0.02)
p3 = (0.3, 0.1, 0.13, 0.1, 0.08, 0.02, 0.04, 0.05, 0.1, 0.12, 0.17, 0.04)
p4 = (0.2, 0.16, 0.12, 0.15, 0.09, 0.01, 0.05, 0.05, 0.1, 0.11, 0.17, 0.05)
p5 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)
p6 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)

s1 = (0.43, 0.13, 0.03, 0.02, 0.01, 0.04, 0.04, 0.04, 0.05, 0.04, 0.03, 0.01)
s2 = (0.3, 0.22, 0.15, 0.01, 0.02, 0.08, 0.12, 0.10, 0.03, 0.02, 0.01, 0.04)
s3 = (0.26, 0.22, 0.22, 0.13, 0.15, 0.016, 0.12, 0.18, 0.3, 0.22, 0.15, 0.01)
s4 = (0.47, 0.1, 0.03, 0.01, 0.02, 0.05, 0.09, 0.04, 0.02, 0.03, 0.03, 0.01)
s5 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)
s6 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)

n1 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n2 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n3 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n4 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n5 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n6 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")

a = 0
b = 0
c = 0
d = 0
e = 0
f = 0

lista = []

while a < len(n1):
 while b < len(n2):
  while c < len(n3):
   while d < len(n4):
    while e < len(n5):
     while f < len(n6):
      lista.append([((p1[a] * p2[b] * p3[c] * p4[d] * p5[e] * p6[f]) / (s1[a] * s2[b] * s3[c] * s4[d] * s5[e] * s6[f])), n1[a] + "  " + n2[b] + "  " + n3[c] + "  " + n4[d] + "  " + n5[e] + "  " + n6[f]])
      f += 1
     f = 0
     e += 1
    e = 0
    d += 1
   d = 0
   c += 1
  c = 0
  b += 1
 b = 0
 a += 1
Reply
#2
According to my calculations, you're trying to build a 569 MegaBytes list. This could be the cause of the memory error.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Understanding and debugging memory error crashes with python3.10.10 Arkaik 5 1,977 Apr-18-2023, 03:22 AM
Last Post: Larz60+
  Memory Error While generating cheksum no mg24 2 984 Sep-25-2022, 10:33 PM
Last Post: mg24
  Gui slot machine-out of memory error steve_shambles 12 4,989 May-18-2020, 11:31 PM
Last Post: deanhystad
  memory error using permutation list of 11 elements kikidog 1 3,846 Sep-10-2019, 08:22 PM
Last Post: ichabod801
  Fix Memory Error while installing a library for Qgis alexastorga 0 2,557 Apr-13-2018, 04:54 PM
Last Post: alexastorga
  Dictionaries and memory error. jarrod0987 5 5,452 Feb-23-2018, 12:15 PM
Last Post: buran

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020