Python Forum
Filling NaNs in a financial dataset
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Filling NaNs in a financial dataset
#9
(Jun-05-2023, 09:14 AM)snippsat Wrote:
(Jun-04-2023, 06:11 PM)larzz Wrote: Yeah I did some additions. Do you think you can help if I upload that one?
Maybe,but not sure as it hard a task when DataFrame is 500 columns wide.
Than at least i or someone else can may take a look at look it.
If you could make the task wanted in smaller scale,it would be fine.
A example of this could be like this.
import pandas as pd

df = pd.read_clipboard()
'''
          A         B         C
0 -0.166919  0.979728 -0.632955
1 -0.297953 -0.912674 -1.365463
2 -0.120211 -0.540679 -0.680481
3       NaN -2.027325  1.533582
4       NaN       NaN  0.461821
5 -0.788073       NaN       NaN
6 -0.916080 -0.612343       NaN
7 -0.887858  1.033826       NaN
8  1.948430  1.025011 -2.982224
9  0.019698 -0.795876 -0.046431
'''
So here have DataFrame with some NaN values,an want to fill in with average values that columns has.
>>> df
          A         B         C
0 -0.166919  0.979728 -0.632955
1 -0.297953 -0.912674 -1.365463
2 -0.120211 -0.540679 -0.680481
3       NaN -2.027325  1.533582
4       NaN       NaN  0.461821
5 -0.788073       NaN       NaN
6 -0.916080 -0.612343       NaN
7 -0.887858  1.033826       NaN
8  1.948430  1.025011 -2.982224
9  0.019698 -0.795876 -0.046431
>>> df.mean()
A   -0.151121
B   -0.231291
C   -0.530307
dtype: float64
>>> df.fillna(df.mean())
          A         B         C
0 -0.166919  0.979728 -0.632955
1 -0.297953 -0.912674 -1.365463
2 -0.120211 -0.540679 -0.680481
3 -0.151121 -2.027325  1.533582
4 -0.151121 -0.231291  0.461821
5 -0.788073 -0.231291 -0.530307
6 -0.916080 -0.612343 -0.530307
7 -0.887858  1.033826 -0.530307
8  1.948430  1.025011 -2.982224
9  0.019698 -0.795876 -0.046431

I have a bigger file that isn't allowed to be uploaded through this site so I did it like this:

https://file.io/cNtj51B5Tswo

If it isn't allowed you can just remove it. This contains 1000 rows of the dataframe I am currently using. I understand that making the task on a smaller scale would be easier, but it shouldn't be that hard to upscale right?
Reply


Messages In This Thread
Filling NaNs in a financial dataset - by larzz - Jun-01-2023, 09:40 PM
RE: Filling NaNs in a financial dataset - by larzz - Jun-03-2023, 10:51 AM
RE: Filling NaNs in a financial dataset - by larzz - Jun-04-2023, 07:56 AM
RE: Filling NaNs in a financial dataset - by larzz - Jun-04-2023, 06:11 PM
RE: Filling NaNs in a financial dataset - by larzz - Jun-05-2023, 12:09 PM
RE: Filling NaNs in a financial dataset - by larzz - Jun-06-2023, 08:31 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  how to extract financial data from photocopy of document angela1 6 3,868 Feb-15-2020, 05:50 PM
Last Post: jim2007
  Financial Modeling MarkHaversham 2 4,850 Feb-11-2020, 10:55 AM
Last Post: Mikhail_Shi
  Google Financial Client ian 7 6,500 Sep-21-2017, 07:23 PM
Last Post: Larz60+
  Filling in missing values melm0 4 5,380 Aug-09-2017, 03:59 PM
Last Post: radioactive9

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020