Posts: 221
Threads: 71
Joined: Dec 2017
Hi All,
lines in files are as below
"11-03-2019_18:43:14,User-name=4841605613,NAS-IP-Address=10.124.113.246,Framed-IP-Address=,Calling-Station-Id=14:a7:2b:03:5f:23,Reply-Mesasge=Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;"
I want to find
11-03-2019 , 18:43:14 , 4841605613 , 10.124.113.246, 14:a7:2b:03:5f:23 , Authentication Failed due to Invalid Password
My regular expression is as below
r = re.compile(r"^(\d{2}\D\d{2}\D\d{4})\D(\d{2}\D\d{2}\D\d{2})\D\w+\D\w+\D([0-9]{10})\D\w+\D\w+\D\w+\D(\d+\.\d+\.\d+\.\d+)\D\w+\D\w+\D\w+\D*([0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2}:[0-9a-f]{2})\D\w+\D\w+\D(\w*\s\w.*)",re.MULTILINE)Output is
Output: ('11-03-2019', '18:43:14', '4841605613', '10.124.113.246', '14:a7:2b:03:5f:23', 'Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;')
however unwanted lines are printing.
like :- ,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;')[/output]
Posts: 1,936
Threads: 8
Joined: Jun 2018
If you want use this data later on, maybe it is useful to read into list of dictionaries for easy analysing?
Processing every row something like this:
>>> s = "11-03-2019_18:43:14,User-name=4841605613,NAS-IP-Address=10.124.113.246,Framed-IP-Address=,Calling-Station-Id=14:a7:2b:03:5f:23,Reply-Mesasge=Authentication Failed due to Invalid Password,NAS-Port-Id=slot=3;subslot=2;port=100;vlanid=2160;vlanid2=1284;"
>>> d = dict()
>>> for i, field in enumerate(s.split(',')):
... if i == 0:
... d['date'], d['time'] = field.split('_')
... else:
... try:
... name, value = field.split('=')
... if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']:
... d.update({name: value})
... except ValueError:
... continue
...
>>> d
{'date': '11-03-2019',
'time': '18:43:14',
'User-name': '4841605613',
'NAS-IP-Address': '10.124.113.246',
'Calling-Station-Id': '14:a7:2b:03:5f:23',
'Reply-Mesasge': 'Authentication Failed due to Invalid Password'}
I'm not 'in'-sane. Indeed, I am so far 'out' of sane that you appear a tiny blip on the distant coast of sanity. Bucky Katt, Get Fuzzy
Da Bishop: There's a dead bishop on the landing. I don't know who keeps bringing them in here. ....but society is to blame.
Posts: 2,015
Threads: 9
Joined: May 2017
Mar-12-2019, 09:05 AM
(This post was last modified: Mar-12-2019, 09:06 AM by DeaD_EyE.)
Don't use reqex for this task.
A solution without using regex and relying that the output format does not change.
def parse(line):
date, user, nas_ip, frame_ip, station, reply, port = line.split(',')
date = datetime.datetime.strptime(date, '%d-%m-%Y_%H:%M:%S')
user = user.replace('User-name=', '')
nas_ip = nas_ip.replace('NAS-IP-Address=', '')
frame_ip = frame_ip.replace('Framed-IP-Address=', '')
station = station.replace('Calling-Station-Id=', '')
reply = reply.replace('Reply-Mesasge=', '')
port = port.replace('NAS-Port-Id=', '').rstrip(';')
port_attr = {}
for item in port.split(';'):
key, value = item.split('=')
port_attr[key] = int(value)
return {
'date': date, 'user': user,
'nas_ip': nas_ip, 'frame_ip': frame_ip,
'station': station, 'reply': reply,
'port': port_attr,
}Output: {'date': datetime.datetime(2019, 3, 11, 18, 43, 14),
'user': '4841605613',
'nas_ip': '10.124.113.246',
'frame_ip': '',
'station': '14:a7:2b:03:5f:23',
'reply': 'Authentication Failed due to Invalid Password',
'port': {'slot': 3,
'subslot': 2,
'port': 100,
'vlanid': 2160,
'vlanid2': 1284}}
Maybe the parsing of datetime is not right. Check the format. Maybe month and day is swapped.
Posts: 221
Threads: 71
Joined: Dec 2017
Mar-12-2019, 10:40 AM
(This post was last modified: Mar-12-2019, 11:07 AM by anna.)
Hi All,
My Code as below, input lines are not same always, -rbash-4.1$ in line is creating problem. I am breaking loop if -rbash-4.1$ found. However missing something.
Input lines
11-03-2019_16:04:10,User-name=0201407256@ttml,NAS-IP-Address=10.124.117.180,Framed-IP-Address=,Calling-Station-Id=00:17:7c:92:a9:d7,Reply-Mesasge=Authentication Success,NAS-Port-Id=pppoe lag-50:3984.2000
12-03-2019_04:05:13,User-name=0201407256@ttml,NAS-IP-Address=10.124.117.180,Framed-IP-Address=,Calling-Station-Id=00:17:7c:92:a9:d7,Reply-Mesasge=Authentication Success,NAS-Port-Id=pppoe lag-50:3984.2000
-rbash-4.1$
import time
import re
import os
import sys
dn_details = sys.argv[1]+'@ttml'
host = '10.xx.113.xx'
username = 'xxxops'
password = 'abcd@1234'
port = 22
#print(dn_details)
remote_conn_pre = paramiko.SSHClient()
remote_conn_pre
remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())
remote_conn_pre.connect(host, username=username, password=password, look_for_keys=False, allow_agent=False)
remote_conn = remote_conn_pre.invoke_shell()
output = remote_conn.recv(1000)
#print(output)
remote_conn.send("\n")
remote_conn.send("grep "+dn_details+" transaction2019-03-11_120000_881.log\n")
time.sleep(5)
output = remote_conn.recv(65535)
remote_conn_pre.close()
for line in output.decode('utf-8').split('\n')[4:]:
d = dict()
for i, field in enumerate(line.split(',')):
if '-rbash-4.1$' in field:
break
else:
if i == 0:
d['date'], d['time'] = field.split('_')
else:
try:
name, value = field.split('=')
if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']:
d.update({name: value})
except ValueError:
continue
print(d['date'],d['time'],d['User-name'],d['NAS-IP-Address'],d['Calling-Station-Id'],d['Reply-Mesasge'])Output: 11-03-2019 16:04:10 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success
12-03-2019 04:05:13 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success
Traceback (most recent call last):
File "crmsessionlogs.py", line 45, in <module>
d['date'], d['time'] = field.split('_')
ValueError: not enough values to unpack (expected 2, got 1)
its working now, is this correct.
import paramiko
import time
import re
import os
import sys
dn_details = sys.argv[1]+'@ttml'
host = '10.124.113.16'
username = 'ttmlops'
password = 'Ttml@1234'
port = 22
#print(dn_details)
remote_conn_pre = paramiko.SSHClient()
remote_conn_pre
remote_conn_pre.set_missing_host_key_policy(paramiko.AutoAddPolicy())
remote_conn_pre.connect(host, username=username, password=password, look_for_keys=False, allow_agent=False)
remote_conn = remote_conn_pre.invoke_shell()
output = remote_conn.recv(1000)
#print(output)
remote_conn.send("\n")
remote_conn.send("grep "+dn_details+" transaction2019-03-11_120000_881.log\n")
time.sleep(5)
output = remote_conn.recv(65535)
remote_conn_pre.close()
for line in output.decode('utf-8').split('\n')[4:]:
if '-rbash-4.1$' in line:
break
else:
d = dict()
for i, field in enumerate(line.split(',')):
# if '-rbash-4.1$' in field:
#break
#else:
if i == 0:
d['date'], d['time'] = field.split('_')
else:
try:
name, value = field.split('=')
if name in ['User-name', 'NAS-IP-Address', 'Calling-Station-Id', 'Reply-Mesasge']:
d.update({name: value})
except ValueError:
continue
print(d['date'],d['time'],d['User-name'],d['NAS-IP-Address'],d['Calling-Station-Id'],d['Reply-Mesasge'])Output: 11-03-2019 16:04:10 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success
12-03-2019 04:05:13 0201407256@ttml 10.124.117.180 00:17:7c:92:a9:d7 Authentication Success
|
