Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 summing rows/columns more quickly
#1
Hi, I recently had an assignment whereby we had to get sums of different columns. The original number of columns was 64 and we wanted to reduce it to 16 by summing up various columns. Thus, each row is a 1x64 array, essentially or you can think of it as an 8x8 matrix. To get the first 8 new sums, I simply took groups of 8, got their sums, and those were my first eight new columns/features. But, it seemed more difficult to get the next 8 sums. These might be considered the column sums if you're thinking of an 8x8 matrix. I ended up writing some pretty round-about code to get what I wanted but I was wondering if anyone had tips on how this code could be shortened to be more simplistic and quicker. Keeping in mind, I did this in pandas. My original code is below. Any tips or hints are much appreciated.

for index, row in df.iloc[:,0:64].iterrows():
    row=np.array(row)
    row=row.reshape(8,8)
    row_sum=row.sum(axis=0)
    row_sum=pd.DataFrame(row_sum.reshape(1,8),columns=[i for i in range(0,8)])
    column_sum=row.sum(axis=1)
    column_sum=pd.DataFrame(column_sum.reshape(1,8), columns=[i for i in range(8,16)])
    result = pd.concat([row_sum, column_sum], axis=1)
    df_2=df_2.append(result)
Jon
Quote
#2
for index, row in df.iloc[:,0:64].iterrows():
    row=numpy.array(row)  
    row=row.reshape(8,8)
    row_sum=row.sum(axis=0)
    column_sum=row.sum(axis=1)
    concatenated = np.concatenate((row_sum,column_sum),axis=0)              #################
    result=pandas.DataFrame(concatenated.reshape(1,16), columns=[i for i in range(0,16)])#############
    df_2=df_2.append(result)
i concatenated row_sum & column_sum 1st, then called Dataframe() once... did this in python 2.7 just in case not working
swallow osama bin laden
Quote
#3
Appreciate that! I only asked because I sent this code to my prof and he said it was overly complex and slow. Not sure what he had in mind or what other method he expected us to use. Writing it this way made sense to me as well from the standpoint of just being able to follow what the code is doing. I'll try and find out and if he explains it, I may post back.
Quote
#4
nice,i'd like to see your next post
swallow osama bin laden
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Pandas: summing columns conditional on the column labels ddd2332 0 190 Sep-10-2020, 05:58 PM
Last Post: ddd2332
  read_csv error and rows/columns missing karlito 9 827 Nov-11-2019, 06:48 AM
Last Post: karlito
  Drop rows if a set of columns has a value dervast 1 504 Sep-12-2019, 04:18 PM
Last Post: sd_0912
  display graph in columns and rows william888 1 546 Jul-02-2019, 10:19 AM
Last Post: dataman
  Imported csv, column text converted to number but not summing Ecniv 4 648 Jun-21-2019, 02:03 PM
Last Post: Ecniv
  Dropping all rows of multiple columns after the max of one cell Thunberd 2 1,091 Jun-01-2018, 10:18 PM
Last Post: Thunberd
  Get rows with same value from dataframe of particular columns angelwings 1 1,080 Apr-11-2018, 02:40 AM
Last Post: scidam
  Stack dataframe columns into rows klllmmm 0 1,520 Sep-03-2017, 02:26 AM
Last Post: klllmmm

Forum Jump:


Users browsing this thread: 1 Guest(s)