Using .append() with list vs dataframe

[Mark17]

My program adds rows to a dataframe by using .append() with a dictionary. In order to make this work, I think I had to do df = df.append() rather than just df.append(). Going back and retrying this afternoon, though, I think I've seen it work either way?

Here, though:

colors = []
for i in summary_results['ROI%']:
    if i < 0:
        colors = colors.append('r')
    else:
        colors = colors.append('g')
print(colors)

I get AttributeError: 'NoneType' object has no attribute 'append' . I need just colors.append('r') rather than assignment.

What's going on with when and why .append() may or may not require assignment (for lists vs. dataframes) in order to work?

[deanhystad]

DataFrame.append(other, ignore_index=False, verify_integrity=False, sort=False)[source]¶
Append rows of other to the end of caller, returning a new object.

list.append(x)
Add an item to the end of the list. Equivalent to a[len(a):] = [x].

Notice that DataFrame.append says it returns a new object. Python.list.append says nothing about a return value, and it returns None.

The word is the same. Use is similar. Results are completely different.

[Mark17]

DataFrame.append(other, ignore_index=False, verify_integrity=False, sort=False)[source]¶
Append rows of other to the end of caller, returning a new object.

list.append(x)
Add an item to the end of the list. Equivalent to a[len(a):] = [x].

Notice that DataFrame.append says it returns a new object. Python.list.append says nothing about a return value, and it returns None.

The word is the same. Use is similar. Results are completely different.

With regard to list.append, does the AttributeError then mean you can't assign that to a variable because there's nothing to assign since nothing is returned?

And with regard to DataFrame.append, if I don't assign the new object to a variable then what happens to it? This...

import pandas as pd
a = [['blue', 3], ['red', 8], ['green', 0]]
df = pd.DataFrame(a)
print(df)
print()
df.append([['grey',12]])
print(df)

...prints out the same thing twice. If I do df = df.append([['grey',12]]) then I see the added row.

[Axel_Erfurt]

append is deprecated

import pandas as pd
df = pd.DataFrame([['blue', 3], ['red', 8], ['green', 0]])
print(df)

df2 = pd.DataFrame([['grey',12]])
df = pd.concat([df, df2], ignore_index=True)
print(df)

Output:
       0  1
0   blue  3
1    red  8
2  green  0
       0   1
0   blue   3
1    red   8
2  green   0
3   grey  12

[deanhystad]

import pandas as pd

rg = pd.DataFrame([['red', 0], ['green', 1]])
rgb = rg.append([['blue', 2]])

print(rg, "\n")
print(rgb)

Output:
       0  1
0    red  0
1  green  1

       0  1
0    red  0
1  green  1
0   blue  2

Notice that the dataframe referenced by "rg" is unchanged. It only contains "red" and "green". "rgb" references a new dataframe object that was created by the append() function. This new dataframe contains the original and the new row "blue".

Python lists are different. When you append to a list, the original list is modified to hold the new values.

rg = ["red", "green"]
rgb = rg.append("blue")
print(rg)
print(rgb)

Output:
['red', 'green', 'blue']
None

Notice that the list object referenced by "rg" contains the color "blue". The original list object was modified to contain the appended value. Also notice that "rgb" refrences None. It is common for Python functions to return None when they don't create any objects.

Python objects live only as long as they are referenced. Unreferenced objects get garbage collected and their memory is reused to make new objects. color = "blue" assigns a str object to a variable named "color". Right now there is only one reference to the str object "blue". If I do this, color = 1, the variable color now references an int object. Now the str object "blue" is not referenced by any variables. It's reference count is zero. There is no way for me to use the str object "blue" anymore because I don't have any variables that reference it. In many languages "blue" would continue to exist in limbo, taking up space until the program ends. In Python, assigning color = 1 not only assigns color to reference an int object, it unassigns color to stop referencing the "blue" object. The "blue" object reference count drops to zero, and Python puts it in the garbage collector.

In your example where you do this: df = df.append([['grey',12]]), the variable "df" is reassigned to reference the new dataframe object created by the append() function. The original dataframe object is no longer referenced by any variables, so it gets garbage collected.

Is that clear?

[Axel_Erfurt]

using append I always get a message

Output:
FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.

[Mark17]

Python objects live only as long as they are referenced. Unreferenced objects get garbage collected and their memory is reused to make new objects. color = "blue" assigns a str object to a variable named "color". Right now there is only one reference to the str object "blue". If I do this, color = 1, the variable color now references an int object. Now the str object "blue" is not referenced by any variables. It's reference count is zero. There is no way for me to use the str object "blue" anymore because I don't have any variables that reference it. In many languages "blue" would continue to exist in limbo, taking up space until the program ends. In Python, assigning color = 1 not only assigns color to reference an int object, it unassigns color to stop referencing the "blue" object. The "blue" object reference count drops to zero, and Python puts it in the garbage collector.

In your example where you do this: df = df.append([['grey',12]]), the variable "df" is reassigned to reference the new dataframe object created by the append() function. The original dataframe object is no longer referenced by any variables, so it gets garbage collected.

Is that clear?

That's good stuff. So the answer to my question about where the new dataframe object goes if not assigned is that it disappears into the recycle bin. Thanks!

[Mark17]

using append I always get a message

Output:
FutureWarning: The frame.append method is deprecated and will be removed from pandas in a future version. Use pandas.concat instead.

I saw this but upon closer look it seemed like it was pertaining to appending one dataframe to another. I am appending a dictionary to a dataframe (to add a new row), which is why I thought the deprecation did not apply. I could certainly be wrong.