time difference bettwenn logs - Printable Version +- Python Forum (https://python-forum.io) +-- Forum: Python Coding (https://python-forum.io/forum-7.html) +--- Forum: General Coding Help (https://python-forum.io/forum-8.html) +--- Thread: time difference bettwenn logs (/thread-41154.html) Pages:
1
2
|
time difference bettwenn logs - enkliy - Nov-20-2023 Hello, I have a log file in txt. lets say <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57418 -> 17.57.172.11:443 (TCP) close connection <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57422 -> 17.57.172.11:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57423 -> 17.57.172.11:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57424 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57425 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57426 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57427 -> 17.57.172.11:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57428 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57429 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57430 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57431 -> 96.17.179.45:443 (TCP) <150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57432 -> 17.57.13.65:443 (TCP) <150>Nov 7 07:38:34 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57427 -> 17.57.172.11:443 (TCP) close connection <150>Nov 7 07:38:34 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57433 -> 17.188.182.68:443 (TCP) <150>Nov 7 07:38:35 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57434 -> 17.57.146.88:5223 (TCP) <150>Nov 7 07:38:37 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57435 -> 17.248.211.69:443 (TCP) I want to have a report/ on doing time difference between each line So any help would be great. I have tried with pandas but that not possible... RE: time difference bettwenn logs - snippsat - Nov-20-2023 (Nov-20-2023, 09:25 AM)enkliy Wrote: So any help would be great. I have tried with pandas but that not possible...You should show what you have tried,can do it Pandas but also there has parse the string first. To help first step which should be is to parse time out and make it a time format. >>> import re >>> from datetime import datetime >>> >>> s = '<150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57422 -> 17.57.172.11:443 (TCP)' >>> r = re.search(r'\d{2}:\d{2}:\d{2}', s) >>> time_str = r.group() >>> time_str '07:38:33' >>> datetime.strptime(time_str, '%H:%M:%S') datetime.datetime(1900, 1, 1, 7, 38, 33)When have time format can use timedelta between times. RE: time difference bettwenn logs - rob101 - Nov-20-2023 Personally, I would not use regex in situations where the position of the data is known and unchanging; rather, I'd simply use the index positions. As a simple demonstration: with open("log", mode="r", encoding="UTF-8") as log: for entry, item in enumerate(log, 1): time_stamp = item[12:20] print(f"{entry:02d}:~", time_stamp)To add: as working demonstration of the difference between the time entries: from datetime import datetime time_stack = [] with open("log", mode="r", encoding="UTF-8") as log: for entry, item in enumerate(log, 1): time_str = item[12:20] time_stamp = datetime.strptime(time_str, '%H:%M:%S') time_stack.append(time_stamp) if len(time_stack) > 1: print(f"{entry:02d}:~", time_str, (time_stack[1] - time_stack[0])) time_stack.pop(0) else: print(f"{entry:02d}:~ Start", time_str)
RE: time difference bettwenn logs - enkliy - Nov-20-2023 Many thanks rob it was very easy and simple solution. The approach way I really appreciate your help RE: time difference bettwenn logs - deanhystad - Nov-20-2023 It is possible with pandas, but pandas doesn't provide much help with this task. The format of the log file also makes it difficult for pandas to parse easily. You should include the date when getting the times so periods spanning the end of day compute the correct delta. The log should also include year. from io import StringIO from datetime import datetime log = StringIO( """<150>Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57422 -> 17.57.172.11:443 (TCP) <150>Nov 7 07:38:43 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57423 -> 17.57.172.11:443 (TCP) <150>Nov 7 08:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57424 -> 17.57.13.65:443 (TCP) <150>Nov 8 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57425 -> 17.57.13.65:443 (TCP)""" ) prev = None for line in log: dt = datetime.strptime(line[5:20], "%b %d %H:%M:%S") if prev is not None: print(dt - prev) prev = dt
RE: time difference bettwenn logs - Larz60+ - Nov-21-2023 FIY: A good writup on many aspects of date and time manipulation can be found datetime — Date and Time Value Manipulation RE: time difference bettwenn logs - Pedroski55 - Nov-21-2023 I put the years in, just for fun: # if you want to repeat, reset file seek to zero: log.seek(0) log = StringIO( """<150>2022 Nov 7 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57422 -> 17.57.172.11:443 (TCP) <150>2023 Nov 7 07:38:43 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57423 -> 17.57.172.11:443 (TCP) <150>2023 Nov 7 08:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57424 -> 17.57.13.65:443 (TCP) <150>2024 Nov 8 07:38:33 DrayTek: Local User (MAC=B0-8C-75-C0-FF-8F): 172.16.91.7:57425 -> 17.57.13.65:443 (TCP)""" )Then just do what deanhystad said: prev = None for line in log: dt = datetime.strptime(line[5:25], "%Y %b %d %H:%M:%S") if prev is not None: print(dt - prev) prev = dt # need this to repeat reading the log log.seek(0)
RE: time difference bettwenn logs - enkliy - Nov-21-2023 I was testing with this code and working on displaying only values where the difference greater than 5 minutes but output goes always ...with Start... I imporetd also timedelta from datetime import datetime, timedelta time_stack = [] with open("log.txt", mode="r", encoding="UTF-8") as log: for entry, item in enumerate(log, 1): time_str = item[12:20] time_stamp = datetime.strptime(time_str, '%H:%M:%S') time_stack.append(time_stamp) time_diff = timedelta(minutes=5) if len(time_stack) > 1: diffe = time_stack[1] - time_stack[0] if diffe >= time_diff: print(f"{entry:02d}:~", time_str, diffe) time_stack.pop(0) else: print(f"{entry:02d}:~ Start", time_str) RE: time difference bettwenn logs - rob101 - Nov-21-2023 What does the new log file look like? With the one posted, there are no time difference > 2 seconds, let alone 5 minutes. RE: time difference bettwenn logs - enkliy - Nov-21-2023 This is the output that I have got at lease it had to be empty 02:~ Start 07:38:32 03:~ Start 07:38:32 04:~ Start 07:38:32 05:~ Start 07:38:32 06:~ Start 07:38:32 07:~ Start 07:38:32 08:~ Start 07:38:32 09:~ Start 07:38:33 10:~ Start 07:38:33 11:~ Start 07:38:33 12:~ Start 07:38:33 13:~ Start 07:38:33 14:~ Start 07:38:33 15:~ Start 07:38:33 16:~ Start 07:38:33 17:~ Start 07:38:33 18:~ Start 07:38:33 19:~ Start 07:38:33 20:~ Start 07:38:33 21:~ Start 07:38:34 22:~ Start 07:38:34 23:~ Start 07:38:35 24:~ Start 07:38:37 25:~ Start 07:38:41 26:~ Start 07:38:41 27:~ Start 07:38:47 28:~ Start 07:38:47 |