Python Forum
pandas : problem with conditional filling of a column
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
pandas : problem with conditional filling of a column
#1
Hi !

I'd like to ask for some help. I am a newbie in python so please forgive me if my problem is obvious.
I have this DataFrame :
import pandas as pd
test = pd.DataFrame({'A':[0, 1, 5, 2, 4, 5, 3, 2, 3]}, index = range(9))
test['B']=(test['A']==test['A'].rolling(5, center = True).max())
I want to create an other column 'C' or modify column 'B' so that
- 'True' is replaced by the value in column 'A'
- 'False is replaced by the value of the row above
See below for the result I want.
   
Thank you for your help ;)
Reply
#2
It is not a noob question. Getting a value from the previous row makes it really difficult to use a vectorized solution. There are commands for shifting a column, but I don't think that applies here since you might need to shift many times.

You could write a loop and use a bunch of DataFrame.iloc() calls, but that is really slow. For something like this I would compute the row in python.
import pandas as pd

df = pd.DataFrame([0, 1, 6, 2, 4, 5, 3, 2, 3], columns=["A"])
df["B"] = df["A"].rolling(5, center=True).max()
C = []
prev = df["A"].values[0]
for A, B in zip(df["A"].values, df["B"].values):
    C.append(prev := A if A == B else prev)
df["C"] = C

print(df)
Output:
A B C 0 0 NaN 0 1 1 NaN 0 2 6 6.0 6 3 2 6.0 6 4 4 6.0 6 5 5 5.0 5 6 3 5.0 5 7 2 NaN 5 8 3 NaN 5
This solution suffers in speed for running in Python, but it takes less than a second to do a table with 100,000 rows.
Reply
#3
Thx a lot !
It works perfectly !!!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Python Alteryx QS-Passing pandas dataframe column inside SQL query where condition sanky1990 0 751 Dec-04-2023, 09:48 PM
Last Post: sanky1990
  Python Pandas Syntax problem? Wrong Output, any ideas? Gbuoy 2 938 Jan-18-2023, 10:02 PM
Last Post: snippsat
  How to assign a value to pandas dataframe column rows based on a condition klllmmm 0 851 Sep-08-2022, 06:32 AM
Last Post: klllmmm
  Basic Pandas, obtaining a value from column and row JamesOzone 2 1,095 Jun-30-2022, 07:16 PM
Last Post: jefsummers
  Numpy error while filling up matrix with Characters august 4 1,870 Apr-13-2022, 10:28 PM
Last Post: august
  Float Slider - Affecting Values in Column 'Pandas' planckepoch86 0 1,405 Jan-22-2022, 02:18 PM
Last Post: planckepoch86
  pandas pivot table: How to find count for each group in Index and Column JaneTan 0 3,325 Oct-23-2021, 04:35 AM
Last Post: JaneTan
  Problem in saving .xlsm (excel) file using pandas dataframe in python shantanu97 2 4,328 Aug-29-2021, 12:39 PM
Last Post: snippsat
  Python Pandas: How do I extract all the >1000 data from a certain column? JaneTan 0 1,571 Jul-17-2021, 09:09 AM
Last Post: JaneTan
  Pandas DataFrame combine rows by column value, where Date Rows are NULL rhat398 0 2,125 May-04-2021, 10:51 PM
Last Post: rhat398

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020