Python Forum

Full Version: Go around memory error
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
I am coding a program which stores 12 numbers in a number of lists (each list has 12 numbers) in a new list. That list also contains a string. I want to do arithmetic operations with every possible combination from the lists (with one number from each list).

When I try with combinations from 5 lists, it works out great. But with 6 lists and more, I run into "memoryerror" if I use the Python 32 bit version. It seems like it never stops thinking if I use the 64 bit version. I have tried to use less decimals in the numbers, then it works better. But I would like to have some decimals there. The ultimate goal would actually be to use up to 15 numbers with up to 4 decimals in up to 8 lists! But 12 numbers in 8 lists would also be nice.

I use Python 3.8.0 [MSC v.1916 64 bit (AMD64)] on win32. My OS is Windows 10 64-bit.

I did some research and found people recommending other people with a similar problem to either use chunking, use databases or use a loop instead of a list. I have also thought about another solution which would be writing the program in C, but I don't know if that would work (I can't do it yet).

The question I have for you is: Given some examples of solutions above, what kind of approach would you recommend or do yourself if you'd run into the same problem? It does not have to be any of the above.
I am just asking for a general approach, not that you fix the problem for me.

I will provide you with the code that works (5 lists) and the code that does not work (6 lists).

Thank you so much!

p1 = (0.1, 0.17, 0.11, 0.12, 0.08, 0.01, 0.03, 0.04, 0.1, 0.12, 0.14, 0.05)
p2 = (0.11, 0.22, 0.12, 0.04, 0.28, 0.2, 0.34, 0.15, 0.16, 0.08, 0.02, 0.02)
p3 = (0.3, 0.1, 0.13, 0.1, 0.08, 0.02, 0.04, 0.05, 0.1, 0.12, 0.17, 0.04)
p4 = (0.2, 0.16, 0.12, 0.15, 0.09, 0.01, 0.05, 0.05, 0.1, 0.11, 0.17, 0.05)
p5 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)

s1 = (0.43, 0.13, 0.03, 0.02, 0.01, 0.04, 0.04, 0.04, 0.05, 0.04, 0.03, 0.01)
s2 = (0.3, 0.22, 0.15, 0.01, 0.02, 0.08, 0.12, 0.10, 0.03, 0.02, 0.01, 0.04)
s3 = (0.26, 0.22, 0.22, 0.13, 0.15, 0.016, 0.12, 0.18, 0.3, 0.22, 0.15, 0.01)
s4 = (0.47, 0.1, 0.03, 0.01, 0.02, 0.05, 0.09, 0.04, 0.02, 0.03, 0.03, 0.01)
s5 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)

n1 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n2 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n3 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n4 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n5 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")

a = 0
b = 0
c = 0
d = 0
e = 0

lista = []

while a < len(n1):
 while b < len(n2):
  while c < len(n3):
   while d < len(n4):
    while e < len(n5):
     lista.append([((p1[a] * p2[b] * p3[c] * p4[d] * p5[e]) / (s1[a] * s2[b] * s3[c] * s4[d] * s5[e])), n1[a] + "  " + n2[b] + "  " + n3[c] + "  " + n4[d] + "  " + n5[e]])
     e += 1
    e = 0
    d += 1
   d = 0
   c += 1
  c = 0
  b += 1
 b = 0
 a += 1
p1 = (0.1, 0.17, 0.11, 0.12, 0.08, 0.01, 0.03, 0.04, 0.1, 0.12, 0.14, 0.05)
p2 = (0.11, 0.22, 0.12, 0.04, 0.28, 0.2, 0.34, 0.15, 0.16, 0.08, 0.02, 0.02)
p3 = (0.3, 0.1, 0.13, 0.1, 0.08, 0.02, 0.04, 0.05, 0.1, 0.12, 0.17, 0.04)
p4 = (0.2, 0.16, 0.12, 0.15, 0.09, 0.01, 0.05, 0.05, 0.1, 0.11, 0.17, 0.05)
p5 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)
p6 = (0.11, 0.23, 0.14, 0.34, 0.13, 0.01, 0.05, 0.01, 0.08, 0.07, 0.2, 0.1)

s1 = (0.43, 0.13, 0.03, 0.02, 0.01, 0.04, 0.04, 0.04, 0.05, 0.04, 0.03, 0.01)
s2 = (0.3, 0.22, 0.15, 0.01, 0.02, 0.08, 0.12, 0.10, 0.03, 0.02, 0.01, 0.04)
s3 = (0.26, 0.22, 0.22, 0.13, 0.15, 0.016, 0.12, 0.18, 0.3, 0.22, 0.15, 0.01)
s4 = (0.47, 0.1, 0.03, 0.01, 0.02, 0.05, 0.09, 0.04, 0.02, 0.03, 0.03, 0.01)
s5 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)
s6 = (0.3, 0.22, 0.19, 0.01, 0.05, 0.04, 0.12, 0.12, 0.04, 0.01, 0.01, 0.04)

n1 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n2 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n3 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n4 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n5 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")
n6 = (" 1", " 2", " 3", " 4", " 5", " 6", " 7", " 8", " 9", "10", "11", "12")

a = 0
b = 0
c = 0
d = 0
e = 0
f = 0

lista = []

while a < len(n1):
 while b < len(n2):
  while c < len(n3):
   while d < len(n4):
    while e < len(n5):
     while f < len(n6):
      lista.append([((p1[a] * p2[b] * p3[c] * p4[d] * p5[e] * p6[f]) / (s1[a] * s2[b] * s3[c] * s4[d] * s5[e] * s6[f])), n1[a] + "  " + n2[b] + "  " + n3[c] + "  " + n4[d] + "  " + n5[e] + "  " + n6[f]])
      f += 1
     f = 0
     e += 1
    e = 0
    d += 1
   d = 0
   c += 1
  c = 0
  b += 1
 b = 0
 a += 1
According to my calculations, you're trying to build a 569 MegaBytes list. This could be the cause of the memory error.