Python Forum
Nan values after merging 2 dataframes
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Nan values after merging 2 dataframes
#1
Hello, I have two pandas dataframes that merged correctly using the "left on = " ", right on =" " method, but after the merge, some of my values in one column are getting Nan in it's row. I've confirmed that they're the same type (object) and that spelling errors in the merge aren't at fault.

Could it be because the two columns merging have different amounts of that value? One has over 30, the other has 24. I do need this to work because these column values have results attached to them that need to be used for calculating a difference.

If it is due to differing amounts, is there anything coding wise I can do to balance it out?
Thank you.
Reply
#2
If two dataframes are not the same size, merging will use NaN to fill in for the missing data. If you don't like the NaNs you cand drop the rows with NaN's, but that means you will be throwing away data.

https://pandas.pydata.org/pandas-docs/st...ropna.html

Another option is replace NaN with a different value, usually zero. But that can also be misleading.

https://pandas.pydata.org/pandas-docs/st...illna.html
https://pandas.pydata.org/pandas-docs/st...place.html

No matter what, any calculations performed on columns must be viewed with suspicion. How do the missing values skew the results?
Reply


Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020