Python Forum
How should I process this data?
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
How should I process this data?
#1
I don't need coding help but I would really appreciate some ideas about how to tackle this problem.

I have this dataset:

[Image: QXgRdc7.png]


Now these numbers should be inversely related. So if one goes up, the other should go down. However, this is not always the case. What I'm really looking for are times when there is a movement in the stress level but also the tolerance to move in the same direction or even in the same direction. This is quite easy to do column by column. But I want to process the data in a way that will find times when they're moving and mostly coordinated in the same direction.

I thought about turning each move into a move in terms of percentage (from row to row) because percentage terms is much better than absolute numbers when they're so different.

[Image: pw36pyo.png]

You can now clearly see at 19:35 that they both move in the same direction (going negative). I would like to process the whole data to find the points which are like this but also which points are most congruent across the whole dataset as it might span across multiple rows.

I am absolutely flummoxed.
Reply
#2
Well, start somewhere, maybe the bottom row or the top row. Compare each column 2 and column 3 for 2 rows, see if both increase, or decrease. If so, do whatever it is you want to do.

if row1, column2 > row2, column2 and row1, column3 > row2, column3:
    doSomething()
RockBlok likes this post
Reply
#3
I was about to suggest doing a delta calculation and comparing, but pedroski's method seems much cleaner.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  How would I process this data? SuchUmami 2 566 Nov-12-2023, 10:05 AM
Last Post: SuchUmami
  Reading data from excel file –> process it >>then write to another excel output file Jennifer_Jone 0 1,116 Mar-14-2023, 07:59 PM
Last Post: Jennifer_Jone
  Process Data from one csv file and write to another CSV file specific column ajin9581 1 2,010 Jun-17-2020, 06:00 PM
Last Post: buran
  How to sharing object between multiple process from main process using Pipe Subrata 1 3,678 Sep-03-2019, 09:49 PM
Last Post: woooee
  Receive data from Redis RQ worker process freak14 0 1,905 Jul-15-2019, 12:39 PM
Last Post: freak14
  Avoid output buffering when redirecting large data (40KB) to another process Ramphic 3 3,418 Mar-10-2018, 04:49 AM
Last Post: Larz60+
  Automating a Data Extraction Process Harrison 12 8,677 Mar-31-2017, 10:44 AM
Last Post: Harrison

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020