Compiler fault or some kind of weird referencing bug?

[Joseph_f2]

So below is my Python code. I'm just messing around with what I was going to use as base for a new project. But after only 2 minutes of writing, I hit a wall. I've been trying various methods to fix it, but I have failed so far unfortunately. Any help at all would be appreciated. The program itself is meant to store the results and then let those be accessed afterwards. In other words, the first things printed should match the second things printed.

It should be easy to understand once you run the code I'm sure, as for the problem, maybe not so. I added extra comments to the code, although it is not my usual practice. Equally, variable names are somewhat poor as I just want to make things work properly first. I've been programming in more languages than I can count for over 8 years now. But I suppose my Python must be rusty.

# Imports
from pprint import pprint

data = []
results = []

# XOR each piece of data based on the value of 'x'
def xorEncrypt(data, x):    
    for i in range(0, len(data)):
        data[i] = data[i] ^ x
    return data

# Fill up the data list with 0 to 15 - 16 entres in total
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15]
for x in range(0, 16):
    data.append(x)

# XOR 'encrypt' using the value of 'i' and add the results to a list (called
# 'results'). Plus, print the result with each iteration.
for i in range(0, 10):    
    result = xorEncrypt(data, i)    
    print("Result:", result)
    results.append(result)
    

# Break-line and heading
print("------------------------------------------------------------------")
print("Final reuslts:")

# Pretty print the results
pprint(results)

A photo of the output on my machine. A friend also got the same result using a brand new installation of Python on a fresh machine.
picpaste. com/pics/1sBgcjGJ.1494201640.png - It wouldn't like me post an image link with my first post, so I've spaced out the URL.

[micseydel]

You only ever create two lists. On line 23, when you append(), you're always appending the same object. Python passes around object references, as you mentioned, if you want copies of objects you have to be explicit (otherwise lots of operations would be really inefficient). The idiom for a shallow copy would be to do

results.append(result[:])

on line 23.

[Joseph_f2]

You only ever create two lists. On line 23, when you append(), you're always appending the same object. Python passes around object references, as you mentioned, if you want copies of objects you have to be explicit (otherwise lots of operations would be really inefficient). The idiom for a shallow copy would be to do

results.append(result[:])

on line 23.

Thank you very much for the help. It worked perfectly :) But could you explain why that works exactly and how you found out that you needed to do that in the first place?

[nilamo]

It's just an oddity of how lists work, as one of the few data types that's mutable.  result and data are both labels referring to the same list, so changing one changes the other.  You append that list to results, and then mutate the list some more... which changes results since it's still the same list.

Slicing it result[:] forces a copy, which breaks all ties to data, so future changes don't effect result or results.  You can also call list() on it, or tuple() to make it very clear that you intend it to be immutable.

>>> x = list(range(5))
>>> x
[0, 1, 2, 3, 4]
>>> y = x
>>> x[1] = 100
>>> x
[0, 100, 2, 3, 4]
>>> y
[0, 100, 2, 3, 4]
>>> y = list(x)
>>> x[2] = 200
>>> x
[0, 100, 200, 3, 4]
>>> y
[0, 100, 2, 3, 4]
>>> y = tuple(x)
>>> x
[0, 100, 200, 3, 4]
>>> y
(0, 100, 200, 3, 4)
>>> x[3] = 300
>>> x
[0, 100, 200, 300, 4]
>>> y
(0, 100, 200, 3, 4)

[micseydel]

It's just an oddity of how lists work, as one of the few data types that's mutable.

Python provides no way to make things immutable yourself (though you can jump through hoops to approximate it), so any user-created class (including third-party modules) has this behavior. I wouldn't call it an "oddity" since it really is fundamental.

Also note that list[:] creates a shallow copy; your list contains a bunch of references to objects, so if the objects are mutable then you could run into similar issues. There's a deepcopy function for the generic case, though [:] is a common idiom for lists.

[volcano63]

It's just an oddity of how lists work, as one of the few data types that's mutable.  

It's not an oddity - Python passes objects by reference (whoever comes form C/C++ background); assignment operator with an object on the right hand does not create a new object - it creates a new reference to an existing object (I wish someone has explained that to me a decade ago   ).

Whoever is used to writing C++, knows to create a copy constructor for an assignment operation (wow, my memory still serves me well ). list[:] may be interpreted as an equivalent of a shallow copy constructor.

[buran]

I wish someone has explained that to me a decade ago Cry

didn't they? or you, finding it by yourself? I mean, I think that 'gotcha' is one of the first that bytes newcomers when start to work with lists...

[volcano63]

didn't they? or you, finding it by yourself?

A long and windy road

Resources were scarce a decade ago, and I was working mostly alone.

I mean, I think that 'gotcha' is one of the first that bytes newcomers when start to work with lists...

bytes, I presume, programmers' Freudian

[buran]

Ups, yeah, mistake :-)

[volcano63]

results.append(result[:])

on line 23.

After a second look at the code, I think the best solution to the specific issue will be to return a new encoded list - instead of changing the argument list in place (which is side-effect - considered bad practice?! )

def xorEncrypt(data, x):
   return [d ^ x for d in data]