Python Forum
Trying to get counts/sum/percentages from pandas similar to pivot table
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Trying to get counts/sum/percentages from pandas similar to pivot table
#1
I am restructuring this question as i have moved away from trying the JSON file since we dont have a server to host this page, and more trouble than its worth.. Since my python script already generates a CSV with raw data, im need to try and reproduce what is currently being done in excel/pivot tables, all within the python script so i can generate a HTML page with the appropriate data.

I have a script that generates a csv file with thousands of records, those are read into an excel file to produce a dashboard which is providing the following:
Total Count of locations (Number)
Total Percentage Up (Decimal)
Count Locations Down (Number)


The above is being calculated and displayed in excel via pivot table. The python script generate a CSV file with the following data:

192.00.1.00,0,"10/06/2023,09:37:35"

IP, Status, Date, Time
Currently that CSV is used as a data source in a excel file, in that file, i have a tab with a reference list of our locations mapped to their IP, so that i can build out the pivot table.

The pivot table looks like this:

Location	Down	Up	Grand Total  Percentage(calculated field)	
7	            3	        5	    8	               38%
9	            2	        9	    11	                18%
Since everything is in excel for the Dashboard, i have formulas to display what we want to see at a high level.

This gives me the total individual registers
=COUNTIF(StoreRegisterBreakdown!A2:A3037, "<>")
This gives me the overall chain total of registers with UP as the status
=(GETPIVOTDATA("Column1",PercentDown!$A$1,"Column2","Up")/B4)
This uses the calculated column to give me a total count of locations with more than 20%
=COUNTIF(PercentDown!E3:E274,">=20%")
This used the calculated column to give me a total count of locations with more than 50%
=COUNTIF(PercentDown!E3:E274,">=50%")

With that said, Since i have the raw data, i know i need to create a dictionary or list in my python code to do the store location reference, but are the above formulas possible all within python? If they are, what should i be looking up online to help accomplish this.. I mean i know its just math, but can the same conditions be setup in python to return the same data?

Id like to be able to handle it all in python so that once the script runs, the data is instantly available and no excel pivot needs to be updated or external source is involved in updating the dashboard.
Reply


Messages In This Thread
Trying to get counts/sum/percentages from pandas similar to pivot table - by cubangt - Sep-15-2023, 02:41 PM

Possibly Related Threads…
Thread Author Replies Views Last Post
  Converting column of values into muliple columns of counts highland44 0 282 Feb-01-2024, 12:48 AM
Last Post: highland44
  Using pyodbc&pandas to load a Table data to df tester_V 3 859 Sep-09-2023, 08:55 PM
Last Post: tester_V
  python pandas sql table with header mg24 3 2,026 Dec-08-2022, 08:31 PM
Last Post: Larz60+
  Need Help! Pandas EXCEL PIVOT psb3958 1 972 Nov-13-2022, 10:37 PM
Last Post: deanhystad
  Read All Emails from Outlook and add the word counts to a DataFrame sanaman_2000 0 1,898 Sep-15-2022, 07:32 AM
Last Post: sanaman_2000
  For Word, Count in List (Counts.Items()) new_coder_231013 6 2,677 Jul-21-2022, 02:51 PM
Last Post: new_coder_231013
  group by create pivot table python dawid294 1 1,322 Jun-22-2022, 06:13 PM
Last Post: Larz60+
  PyPi Download Counts and Warehouse Migration? tlee0818 1 1,319 Mar-20-2022, 07:41 PM
Last Post: snippsat
  Sum the values in a pandas pivot table specific columns klllmmm 1 4,696 Nov-19-2021, 04:43 PM
Last Post: klllmmm
  pandas pivot table: How to find count for each group in Index and Column JaneTan 0 3,362 Oct-23-2021, 04:35 AM
Last Post: JaneTan

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020