Python Forum
time difference bettwenn logs - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: time difference bettwenn logs (/thread-41154.html)

Pages: 1 2

RE: time difference bettwenn logs - rob101 - Nov-21-2023

If we assume this log file:
Output:
<150>Nov  7 07:38:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57418 -> 17.57.172.11:443 (TCP) close connection
<150>Nov  7 07:39:10 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57422 -> 17.57.172.11:443 (TCP)
<150>Nov  7 07:44:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57423 -> 17.57.172.11:443 (TCP)
<150>Nov  7 07:46:20 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57424 -> 17.57.13.65:443 (TCP)
<150>Nov  7 07:48:30 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57425 -> 17.57.13.65:443 (TCP)
<150>Nov  7 07:50:10 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57426 -> 17.57.13.65:443 (TCP)
<150>Nov  7 07:55:10 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57427 -> 17.57.172.11:443 (TCP)
<150>Nov  7 08:01:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57428 -> 17.57.13.65:443 (TCP)
<150>Nov  7 08:06:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57429 -> 17.57.13.65:443 (TCP)
<150>Nov  7 08:08:30 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57430 -> 17.57.13.65:443 (TCP)
<150>Nov  7 08:10:55 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57431 -> 96.17.179.45:443 (TCP)
<150>Nov  7 08:15:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57432 -> 17.57.13.65:443 (TCP)
<150>Nov  7 08:22:25 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57427 -> 17.57.172.11:443 (TCP) close connection
<150>Nov  7 08:27:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57433 -> 17.188.182.68:443 (TCP)
<150>Nov  7 08:35:45 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57434 -> 17.57.146.88:5223 (TCP)
<150>Nov  7 08:38:00 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57435 -> 17.248.211.69:443 (TCP)
... (which I've simply adapted from the one posted) and use this modified script:
from datetime import datetime, timedelta

time_stack = []

with open("log2", mode="r", encoding="UTF-8") as log:
    for entry, item in enumerate(log, 1):
        time_str = item[12:20]
        time_stamp = datetime.strptime(time_str, '%H:%M:%S')
        time_stack.append(time_stamp)
        if len(time_stack) > 1:
            diff = time_stack[1] - time_stack[0]
            if diff >= timedelta(seconds=300):
                print(f"{entry:02d}:~", time_str, diff)
            time_stack.pop(0)
        else:
            print(f"{entry:02d}:~ Start", time_str)
... we get this report:

Output:
01:~ Start 07:38:00
07:~ 07:55:10 0:05:00
08:~ 08:01:00 0:05:50
09:~ 08:06:00 0:05:00
13:~ 08:22:25 0:07:25
15:~ 08:35:45 0:08:45
... which shows only entries that have a time difference of five minutes or more (5 minutes = 300 seconds).

Is that what you're looking for?

RE: time difference bettwenn logs - enkliy - Nov-21-2023

Yes Rob this is what I needed. I made a mistake doing delta with minutes...

RE: time difference bettwenn logs - rob101 - Nov-21-2023

(Nov-21-2023, 03:36 PM)enkliy Wrote: Yes Rob this is what I needed. I made a mistake doing delta with minutes...

No worries. You'll get the hang of it; just keep tying.

RE: time difference bettwenn logs - deanhystad - Nov-21-2023

Quote: I made a mistake doing delta with minutes...
No, you did not. You can use minutes to initialize a timedelta. The two timedelta objects below are equal.
from datetime import timedelta

minutes = timedelta(minutes=5)
seconds = timedelta(seconds=300)
print(minutes == seconds)
Your problem was here:
        if  len(time_stack) > 1:
            diffe = time_stack[1] - time_stack[0]
            if diffe >= time_diff:
               print(f"{entry:02d}:~", time_str, diffe)
               time_stack.pop(0)   # <- pop does not belong here
            else:
                print(f"{entry:02d}:~ Start", time_str)
          # time_stack.pop(0)   pop belongs here
Let's say I have a list of numbers and I want to report when the difference between numbers >= 5. Using your algorithm, this is what happens:
Output:
numbers = [1, 2, 7, 42]
stack = []

Numbers  Stack
      1  [1]
      2  [1, 2]
      7  [1, 2, 7]
     42 [1, 2, 7, 42]
The program thinks the difference is always 1 because stack[1] - stack[0] == 1. Not popping values from the stack when the difference < 5 results adding more and more entries to the stack and never comparing any log entries but the first two.

Using a list for this problem is odd. All you need to remember is the previous time.
from datetime import datetime, timedelta
 
prev_stamp = None
time_diff = timedelta(minutes=5)
with open("log.txt", mode="r", encoding="UTF-8") as log:
    for entry, item in enumerate(log, 1):
        time_str = item[5:20]
        time_stamp = datetime.strptime(time_str, '%b %d %H:%M:%S')
        if prev_stamp:
            diffe = time_stamp - prev_stamp
            if diffe >= time_diff:
               print(f"{entry:02d}:~", time_str, diffe)
            else:
                print(f"{entry:02d}:~ Start", time_str)
        prev_stamp = time_stamp
Throwing away the date information is a mistake. The first log entry for each day results in a negative diffe if there's less than 24 hours between this and the previous entry. If you include the month and day this only happens to the first log entry of the year.

RE: time difference bettwenn logs - rob101 - Nov-21-2023

(Nov-21-2023, 04:33 PM)deanhystad Wrote: Using a list for this problem is odd. All you need to remember is the previous time.

It may seem a little odd to you, but it's simply a demonstration of a different way of doing things. It could could even be extended, or one could use a dictionary object rather than a list object, so that the particulars of any log entry can be recalled without having to once again access the file from storage. It's simply a proof of concept is all.