pandas : problem with conditional filling of a column

pandas : problem with conditional filling of a column - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: pandas : problem with conditional filling of a column (/thread-40382.html)

pandas : problem with conditional filling of a column - Xigris - Jul-20-2023

Hi !

I'd like to ask for some help. I am a newbie in python so please forgive me if my problem is obvious.
I have this DataFrame :

import pandas as pd
test = pd.DataFrame({'A':[0, 1, 5, 2, 4, 5, 3, 2, 3]}, index = range(9))
test['B']=(test['A']==test['A'].rolling(5, center = True).max())

I want to create an other column 'C' or modify column 'B' so that
- 'True' is replaced by the value in column 'A'
- 'False is replaced by the value of the row above
See below for the result I want.
[attachment=2463]
Thank you for your help ;)

RE: pandas : problem with conditional filling of a column - deanhystad - Jul-20-2023

It is not a noob question. Getting a value from the previous row makes it really difficult to use a vectorized solution. There are commands for shifting a column, but I don't think that applies here since you might need to shift many times.

You could write a loop and use a bunch of DataFrame.iloc() calls, but that is really slow. For something like this I would compute the row in python.

import pandas as pd

df = pd.DataFrame([0, 1, 6, 2, 4, 5, 3, 2, 3], columns=["A"])
df["B"] = df["A"].rolling(5, center=True).max()
C = []
prev = df["A"].values[0]
for A, B in zip(df["A"].values, df["B"].values):
    C.append(prev := A if A == B else prev)
df["C"] = C

print(df)

Output:   A    B  C
0  0  NaN  0
1  1  NaN  0
2  6  6.0  6
3  2  6.0  6
4  4  6.0  6
5  5  5.0  5
6  3  5.0  5
7  2  NaN  5
8  3  NaN  5

This solution suffers in speed for running in Python, but it takes less than a second to do a table with 100,000 rows.

RE: pandas : problem with conditional filling of a column - Xigris - Jul-22-2023

Thx a lot !
It works perfectly !!!