Python Forum
summing rows/columns more quickly
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
summing rows/columns more quickly
Hi, I recently had an assignment whereby we had to get sums of different columns. The original number of columns was 64 and we wanted to reduce it to 16 by summing up various columns. Thus, each row is a 1x64 array, essentially or you can think of it as an 8x8 matrix. To get the first 8 new sums, I simply took groups of 8, got their sums, and those were my first eight new columns/features. But, it seemed more difficult to get the next 8 sums. These might be considered the column sums if you're thinking of an 8x8 matrix. I ended up writing some pretty round-about code to get what I wanted but I was wondering if anyone had tips on how this code could be shortened to be more simplistic and quicker. Keeping in mind, I did this in pandas. My original code is below. Any tips or hints are much appreciated.

for index, row in df.iloc[:,0:64].iterrows():
    row_sum=pd.DataFrame(row_sum.reshape(1,8),columns=[i for i in range(0,8)])
    column_sum=pd.DataFrame(column_sum.reshape(1,8), columns=[i for i in range(8,16)])
    result = pd.concat([row_sum, column_sum], axis=1)
for index, row in df.iloc[:,0:64].iterrows():
    concatenated = np.concatenate((row_sum,column_sum),axis=0)              #################
    result=pandas.DataFrame(concatenated.reshape(1,16), columns=[i for i in range(0,16)])#############
i concatenated row_sum & column_sum 1st, then called Dataframe() once... did this in python 2.7 just in case not working
Appreciate that! I only asked because I sent this code to my prof and he said it was overly complex and slow. Not sure what he had in mind or what other method he expected us to use. Writing it this way made sense to me as well from the standpoint of just being able to follow what the code is doing. I'll try and find out and if he explains it, I may post back.
nice,i'd like to see your next post

Possibly Related Threads…
Thread Author Replies Views Last Post
  Pandas: summing columns conditional on the column labels ddd2332 0 467 Sep-10-2020, 05:58 PM
Last Post: ddd2332
  read_csv error and rows/columns missing karlito 9 1,131 Nov-11-2019, 06:48 AM
Last Post: karlito
  Drop rows if a set of columns has a value dervast 1 628 Sep-12-2019, 04:18 PM
Last Post: sd_0912
  display graph in columns and rows william888 1 670 Jul-02-2019, 10:19 AM
Last Post: dataman
  Imported csv, column text converted to number but not summing Ecniv 4 826 Jun-21-2019, 02:03 PM
Last Post: Ecniv
  Dropping all rows of multiple columns after the max of one cell Thunberd 2 1,218 Jun-01-2018, 10:18 PM
Last Post: Thunberd
  Get rows with same value from dataframe of particular columns angelwings 1 1,236 Apr-11-2018, 02:40 AM
Last Post: scidam
  Stack dataframe columns into rows klllmmm 0 1,654 Sep-03-2017, 02:26 AM
Last Post: klllmmm

Forum Jump:

User Panel Messages

Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020