Python Forum
Python3 Regex Help. - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Python3 Regex Help. (/thread-14042.html)

Pages: 1 2


Python3 Regex Help. - Stealthychu - Nov-12-2018

I currently have a script setup to ssh into a network device and extract parameters into a mysql database.

My data base displays the values: Hostname, Model, Image, IOSVersion, CPUModel, CPUSpeed, MACAddress and IPAddress.

Currently the script works to the point where it connects through ssh successfully and extracts the first six values but i believe my code relating to extracting the MAC Address and IP Address are wrong but am un-sure how to fix this.


In mysql table ti displays 'unknown' for both values so i assume i just have the re.search or re in general values wrong but am unsure how to fix it.
Script:

#Extract parameters ONLY from network device

#Send to MYSQL database NetMon

# cpu_speed and cpu_model do not work on 2811 routers in N523

import pymysql as mdb
import pymysql.cursors
#import MySQLdb as mdb
import paramiko
import os.path
import subprocess
import datetime
import time
import sys
import re


#MySQL Database Details

sql_host = "localhost"
sql_username = "admin"
sql_password = "python"
sql_database = "NetMon"



# Connect to MYSQL
def sql_connection(command, values):

    sql_connection = mdb.connect(sql_host,sql_username,sql_password,sql_database)

    cursor = sql_connection.cursor()
    cursor.execute("use NetMon")
    cursor.execute(command, values)

    sql_connection.commit()

#Open ssh connection

ip_address = "192.168.2.101"
username = "admin01"
password = "cisco12345"

ssh_client = paramiko.SSHClient()

ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh_client.connect(hostname=ip_address,username=username,password=password)

remote_connection = ssh_client.invoke_shell()
remote_connection.send(b"terminal length 0\n")
time.sleep(1)


#Reading commands

selected_cisco_commands = b'''show version | include (, Version|uptime is|bytes of memory|Hz)&\
                                  show inventory&\
                                  show interfaces | include bia&\
                                  show processes cpu | include CPU utilization&\
                                  show memory statistics&\
                                  show ip int brief | include (Ethernet|Serial)&\
                                  show cdp neighbors detail | include Device ID&\
                                  show ip protocols | include Routing Protocol'''

#Splitting commands using the "&" = and
command_list = selected_cisco_commands.split(b"&")


for each_line in command_list:
    remote_connection.send(each_line + b'\n')
    time.sleep(3)



#Checking command output for IOS syntax errors
output = remote_connection.recv(65535)

if re.search(b"% Invalid input detected at", output):
    print ("* There was at least one IOS syntax error on device %s")

else:
    print ("* All parameters were extracted from device %s")
print (output.decode('ascii') + "\n")


#Extracting Device Parameters


dev_hostname = re.search(b"(.+) uptime is", output)
hostname = dev_hostname.group(1)
print (hostname.decode('ascii'))



dev_model = re.search(b"(.+?) (.+?) (.+) bytes of memory", output)
model = dev_model.group(2)
print (model.decode('ascii'))

dev_image_name = re.search(b" \((.+)\), Version", output)
image_name = dev_image_name.group(1)
print (image_name.decode('ascii'))

dev_os = re.search(b"\), Version (.+),", output)
os = dev_os.group(1)
print (os.decode('ascii'))

cpu_model = ""
if re.search(b".isco (.+?) \((.+)\) processor(.+)\n", output) == None:
    cpu_model = "unknown"
else:
    cpu_model = re.search(b".isco (.+?) \((.+)\) processor(.+)\n", output).group(2)
print (cpu_model)

cpu_speed = ""
if re.search(b"(.+?)at (.+?)MHz(.+)\n", output) == None:
    cpu_speed = "unknown"
else:
    cpu_speed = re.search(b"(.+?)at (.+?)MHz(.+)\n", output).group(2)
print (cpu_speed)



mac_address = ""
if re.search(b"([0-9a-fA-F]:?){12}\n", output) == None:
    mac_address = "unknown"
else:
    mac_address = re.search(b"([0-9a-fA-F]:?){12}\n", output)
print (mac_address)


ip_address = ""
if re.search(b"[0-9]+(?:\.[0-9]+){3}\n", output) == None:
    ip_address = "unknown"
else:
    ip_address = re.search(b"[0-9]+(?:\.[0-9]+){3}\n", output)
print (ip_address)

#Insert/Update if exists all network devices data into the MySQL database table NetworkDevices. To access login to sql
sql_connection("REPLACE INTO NetworkDevices(Hostname,Model,Image,IOSVersion,CPUModel,CPUSpeed,MACAddress,IPAddress) VALUES(%s, %s, %s, %s, %s, %s, %s, %s)", (hostname, model, image_name, os, cpu_model, cpu_speed, mac_address, ip_address))

poll_timestamp = datetime.datetime.now()


#Closing the SSH connection
ssh_client.close()



RE: Python3 Regex Help. - Gribouillis - Nov-12-2018

Do you have an example of a real "output" byte string from which these data are extracted?


RE: Python3 Regex Help. - Stealthychu - Nov-12-2018

what do you mean by that sorry?

do you mean like the results of entering the commands in the routing device?


RE: Python3 Regex Help. - Gribouillis - Nov-12-2018

I mean that you have many expressions like re.search(b"....", output). I want to know a typical contents of the output variable.


RE: Python3 Regex Help. - Stealthychu - Nov-12-2018

The output variable is a combination of all the commands entered into the network device.
The output is the results given by the router when these commands are entered:
'show inventory'
'show interfaces'
'show processes cpu'
'show memory statistics'
'show ip int brief'
'show cdp neighbours'
'show ip protocols'

in the two i am having issues with they would be using the part of the output which follows:

The output that displays the IP Addresses I want to extract is: 'show ip interface brief' which gives the following output:

[Image: show-ip-int-brief.png]

The output that displays the MAC Address I want to extract is: 'show interfaces'

[Image: show-interfaces.png]


RE: Python3 Regex Help. - Gribouillis - Nov-12-2018

It seems that there is no newline in the string after the ip address, it means that you should probably remove the \n in the regular expression, so use re.search(br"[0-9]+(?:\.[0-9]+){3}", output). Also a good rule is to always use raw strings when writing literal regular expressions in python code, so use br"..." instead of b"..." when you write regular expressions.

I cannot see the mac address in your second picture, where is it? Or perhaps it is the c201.1110.0000 which is one possible form of encoding a mac address as wikipedia says, that is to say three groups of 4 hexadecimal digits, which could be written as the regex br"[0-9a-fA-F]{4}(?:\.[0-9a-fA-F]{4}){2}" but not all mac addresses have this form.


RE: Python3 Regex Help. - Stealthychu - Nov-12-2018

Thanks, for your help so far.

I believe the address I am trying to extract is this one (circled in blue).

[Image: jJTRxq]


RE: Python3 Regex Help. - Gribouillis - Nov-12-2018

(Nov-12-2018, 06:23 PM)Stealthychu Wrote: I believe the address I am trying to extract is this one
See my edits in the previous reply.


RE: Python3 Regex Help. - Stealthychu - Nov-12-2018

I attempted to change the code for the IP address extraction as you suggested to this.

ip_address = ""
if re.search(br"[0-9]+(?:\.[0-9]+){3}", output) == None:
    ip_address = "unknown"
else:
    ip_address = re.search(br"[0-9]+(?:\.[0-9]+){3}", output)
print (ip_address)
But i am no met with this error when i attempt to run it and the sql table still shows the IP Address value as unknown.

[Image: dEEoAA]

I changed it as you suggested with the MAC address as well and it seem i go the same error except with an extra line stating it found a match for the correct MAC address but gave the same errors as before after that.


RE: Python3 Regex Help. - Gribouillis - Nov-12-2018

Then you need to learn python's re module, because re.search() returns a match object and you want a bytes string, so use
ip_address = re.search(br"[0-9]+(?:\.[0-9]+){3}", output).group(0)
The same applies to the rest of the code.