Python Forum
summing rows/columns more quickly
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
summing rows/columns more quickly
#1
Hi, I recently had an assignment whereby we had to get sums of different columns. The original number of columns was 64 and we wanted to reduce it to 16 by summing up various columns. Thus, each row is a 1x64 array, essentially or you can think of it as an 8x8 matrix. To get the first 8 new sums, I simply took groups of 8, got their sums, and those were my first eight new columns/features. But, it seemed more difficult to get the next 8 sums. These might be considered the column sums if you're thinking of an 8x8 matrix. I ended up writing some pretty round-about code to get what I wanted but I was wondering if anyone had tips on how this code could be shortened to be more simplistic and quicker. Keeping in mind, I did this in pandas. My original code is below. Any tips or hints are much appreciated.

for index, row in df.iloc[:,0:64].iterrows():
    row=np.array(row)
    row=row.reshape(8,8)
    row_sum=row.sum(axis=0)
    row_sum=pd.DataFrame(row_sum.reshape(1,8),columns=[i for i in range(0,8)])
    column_sum=row.sum(axis=1)
    column_sum=pd.DataFrame(column_sum.reshape(1,8), columns=[i for i in range(8,16)])
    result = pd.concat([row_sum, column_sum], axis=1)
    df_2=df_2.append(result)
Jon
Reply
#2
for index, row in df.iloc[:,0:64].iterrows():
    row=numpy.array(row)  
    row=row.reshape(8,8)
    row_sum=row.sum(axis=0)
    column_sum=row.sum(axis=1)
    concatenated = np.concatenate((row_sum,column_sum),axis=0)              #################
    result=pandas.DataFrame(concatenated.reshape(1,16), columns=[i for i in range(0,16)])#############
    df_2=df_2.append(result)
i concatenated row_sum & column_sum 1st, then called Dataframe() once... did this in python 2.7 just in case not working
Reply
#3
Appreciate that! I only asked because I sent this code to my prof and he said it was overly complex and slow. Not sure what he had in mind or what other method he expected us to use. Writing it this way made sense to me as well from the standpoint of just being able to follow what the code is doing. I'll try and find out and if he explains it, I may post back.
Reply
#4
nice,i'd like to see your next post
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas: summing columns conditional on the column labels ddd2332 0 364 Sep-10-2020, 05:58 PM
Last Post: ddd2332
  read_csv error and rows/columns missing karlito 9 909 Nov-11-2019, 06:48 AM
Last Post: karlito
  Drop rows if a set of columns has a value dervast 1 543 Sep-12-2019, 04:18 PM
Last Post: sd_0912
  display graph in columns and rows william888 1 582 Jul-02-2019, 10:19 AM
Last Post: dataman
  Imported csv, column text converted to number but not summing Ecniv 4 708 Jun-21-2019, 02:03 PM
Last Post: Ecniv
  Dropping all rows of multiple columns after the max of one cell Thunberd 2 1,133 Jun-01-2018, 10:18 PM
Last Post: Thunberd
  Get rows with same value from dataframe of particular columns angelwings 1 1,132 Apr-11-2018, 02:40 AM
Last Post: scidam
  Stack dataframe columns into rows klllmmm 0 1,566 Sep-03-2017, 02:26 AM
Last Post: klllmmm

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020