Not able to figure out what is wrong with argparse

radioactive9

Hello

I am facing a very unique issue. I have written the following code which will be called by a Tool and pass certain arguments. I was using argpass thinking it will be easy. But I am not able to figure out what is going wrong with it

if __name__ == "__main__":
    log_path = "/var/log/tooly2/"
    log_file = "notifications.log"
    log_file_path = log_path + log_file
    if os.access(log_path, os.W_OK) and os.access(log_path + log_file, os.W_OK):
        # Initialize the default/basic and a custom logger
        logging.basicConfig(filename=log_file_path, level=logging.DEBUG, format='%(process)d %(asctime)s %(levelname)s - %(funcName)s: %(message)s', datefmt="%Y-%m-%d %H:%M:%S")
    else:
        print ("The log directory or the log file is not writable for this script, so no logging is possible. Exiting now.")
        sys.exit(1)
    try:
        n = len(sys.argv)
        logging.debug('Total arguments passed: %s', n) #check the arguments
        logging.debug('I can print the first argument %s', sys.argv[1]) #check arguments 1
        logging.debug('I can print the first argument %s', sys.argv[2]) #check argument 2
        parser = argparse.ArgumentParser()
        parser.add_argument("-N", "--host", help="Node Name") #taken
        parser.add_argument("-I", "--IPAddress", help="Node Name") #taken
        parser.add_argument("-S", "--service", help="Service Name")
        parser.add_argument("-D", "--displayname", help="Display Name")
        parser.add_argument("-G", "--group", help="Group")
        parser.add_argument("-E", "--environment", help="Environement") #Taken
        parser.add_argument("-Z", "--newstate", help="Current Type of the alert")
        parser.add_argument("-A", "--appname", help="Application Name")
        parser.add_argument("-HA", "--hostappname", help="Host App Name") #taken
        parser.add_argument("-P", "--priority", help="Priority") #taken
        parser.add_argument("-O", "--output", help="Description") #taken
        parser.add_argument("-T", "--eventtime", help="Event Time") #taken
        parser.add_argument("-OS", "--os", help="Operating System") #taken
        parser.add_argument("-OP", "--operator", help="Operator") #taken
        parser.add_argument("-NT", "--notificationtype", help="Notification Type Problem or Clear") #taken
        parser.add_argument("-ND", "--notification_default", help="Define which Operator") #taken
        parser.add_argument("-NM", "--groupoperatoros", help="OS Group") #taken
        parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group") #taken
        arguments = parser.parse_args()
        logging.debug('I am not able to reach here')
    except Exception as e:
        logging.debug('This message should go to the log file %s', e )
        exit(1)

In the logs I am getting

Output:5618 2022-03-15 12:36:50 DEBUG - <module>: Total arguments passed: 35
5618 2022-03-15 12:36:50 DEBUG - <module>: I can print the first argument key --host
5618 2022-03-15 12:36:50 DEBUG - <module>: I can print the second argument value ccccccccccc85
5629 2022-03-15 12:37:10 DEBUG - <module>: Total arguments passed: 35
5629 2022-03-15 12:37:10 DEBUG - <module>: I can print the first argument key --host
5629 2022-03-15 12:37:10 DEBUG - <module>: I can print the second argument value xxxxxxxxxx190
5633 2022-03-15 12:37:26 DEBUG - <module>: Total arguments passed: 35

So as you see the line
logging.debug('I am not able to reach here') is not printing which means the code is getting lost somewhere in the argpass

Not sure how to even troubleshoot and figure out the issue

Any direction will be very helpful Wall

**Gribouillis** · Mar-15-2022, 12:39 PM

This code is incomplete, I was unable to reproduce the error.

radioactive9 · (This post was last modified: Mar-15-2022, 01:39 PM by radioactive9.)

(Mar-15-2022, 12:39 PM)Gribouillis Wrote: This code is incomplete, I was unable to reproduce the error.

In the code line 36 doesn't get executed. It doesn't throw any error as well. That is my problem. I do not see any reason why that line will not execute and stop there. There are lot of functions that will be called after that. But for now my problem is it doesn't execute that line 36 and stops there. Doesn't throw error as well

Its not a compilation error as well - as I am getting the output in Logging for line 13 14 15

I intentionally put the try catch block for argparse as I was not able to figure out the problem.

EDIT:

To do further troubleshooting I added the following line in the code

logging.debug('Complete list of argument received %s', sys.argv[1:])

I got the arguments like this. Is this format OK for argparse to parse properly

Quote:20205 2022-03-15 14:30:15 DEBUG - <module>: Complete list of argument received ['--host', 'ccccccccc87', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-User_Role', '--Environment', 'PROD', '--eventtime', '2022-03-15 14:30:14 +0100', '--division', 'Tec', '--operatoros', 'G_WNT_RUN', '--operatormw', 'G_NT_RUN', '--appname', 'TEC-User_Role', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99']

DeaD_EyE · Mar-15-2022, 01:44 PM

Your problem is your debugging code.
The debugging code had a bug. If no arguments are supplied, the index access sys.argv[1] will throw the exception.
Catching all Exceptions with Exception is wrong. Instead, let fail your code in the first place.
Then look which Exceptions could occur and which Exceptions you expect to catch.

import argparse


def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("-N", "--host", help="Node Name")
    parser.add_argument("-I", "--IPAddress", help="Node Name")
    parser.add_argument("-S", "--service", help="Service Name")
    parser.add_argument("-D", "--displayname", help="Display Name")
    parser.add_argument("-G", "--group", help="Group")
    parser.add_argument("-E", "--environment", help="Environement")
    parser.add_argument("-Z", "--newstate", help="Current Type of the alert")
    parser.add_argument("-A", "--appname", help="Application Name")
    parser.add_argument("-HA", "--hostappname", help="Host App Name")
    parser.add_argument("-P", "--priority", help="Priority")
    parser.add_argument("-O", "--output", help="Description")
    parser.add_argument("-T", "--eventtime", help="Event Time")
    parser.add_argument("-OS", "--os", help="Operating System")
    parser.add_argument("-OP", "--operator", help="Operator")
    parser.add_argument(
        "-NT", "--notificationtype", help="Notification Type Problem or Clear"
    )
    parser.add_argument("-ND", "--notification_default", help="Define which Operator")
    parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken
    parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")
    return parser.parse_args()


if __name__ == "__main__":
    args = get_args()
    print(args)


# This error came from line 14
# Line 14: logging.debug('I can print the first argument %s', sys.argv[1]) #check arguments 1
# 14927 2022-03-15 14:18:52 DEBUG - <module>: This message should go to the log file list index out of range

# conclusion, your debugging code introduced new bugs

If you want to use short and long arguments which are required, then you can add required=True.

import argparse


parser = argparse.ArgumentParser()
parser.add_argument("-o", "--output", required=True)

# simulating the commandline
args = parser.parse_args(["-o", "test_OUTPUT"])
print(args)


# This here will raise an SystemExit exception because of wrong input
# usually the program will then exit with the supplied code
try:
    parser.parse_args([])
except SystemExit as e:
    print("Method parse_args wanted to exit the programm with return_code", e.code)

radioactive9 · (This post was last modified: Mar-15-2022, 02:21 PM by radioactive9.)

Thank You

I took your Code and tried to parse my arguments the way the tool is sending and it failed. Because it is not in a expected format

import argparse
 
 
def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("-N", "--host", help="Node Name")
    parser.add_argument("-I", "--IPAddress", help="Node Name")
    parser.add_argument("-S", "--service", help="Service Name")
    parser.add_argument("-D", "--displayname", help="Display Name")
    parser.add_argument("-G", "--group", help="Group")
    parser.add_argument("-E", "--environment", help="Environement")
    parser.add_argument("-Z", "--newstate", help="Current Type of the alert")
    parser.add_argument("-A", "--appname", help="Application Name")
    parser.add_argument("-HA", "--hostappname", help="Host App Name")
    parser.add_argument("-P", "--priority", help="Priority")
    parser.add_argument("-O", "--output", help="Description")
    parser.add_argument("-T", "--eventtime", help="Event Time")
    parser.add_argument("-OS", "--os", help="Operating System")
    parser.add_argument("-OP", "--operator", help="Operator")
    parser.add_argument(
        "-NT", "--notificationtype", help="Notification Type Problem or Clear"
    )
    parser.add_argument("-ND", "--notification_default", help="Define which Operator")
    parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken
    parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")
    return parser.parse_args()
 
 
if __name__ == "__main__":
    args = get_args()
    print(args)

Quote:run Testing.py '--host', 'cccccccccc602', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-Server', '--Environment', 'PROD', '--eventtime', '2022-03-15 15:13:26 +0100', '--division', 'Tec', '--operatoros', 'GSC_W_RUN', '--operatormw', 'GSC_W_RUN', '--appname', 'TEC-Server', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99'

Obviously got error

An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

Error:Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\utils\py3compat.py", line 188, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 37, in <module>
    args = get_args()

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 33, in get_args
    return parser.parse_args()

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 1733, in parse_args
    self.error(msg % ' '.join(argv))

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2389, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2376, in exit
    _sys.exit(status)

SystemExit: 2

Now i cannot change the format in which tool is sending me the data as argument. What are my options

**Gribouillis** · (This post was last modified: Mar-15-2022, 03:00 PM by Gribouillis.)

Try this hack to log the error

parser = argparse.ArgumentParser()
...

_error = parser.error

def error(message):
    logging.debug('argparse parser error %s', message)
    _error(message)

parser.error = error
    
arguments = parser.parse_args()

radioactive9 · Mar-15-2022, 03:08 PM

(Mar-15-2022, 02:57 PM)Gribouillis Wrote: Try this hack to log the error

parser = argparse.ArgumentParser()
...

_error = parser.error

def error(message):
    logging.debug('argparse parser error %s', message)
    _error(message)

parser.error = error
    
arguments = parser.parse_args()

Hi

Do you think my problem is the argument's that is being passed ? It is not in expected format. Problem is tool sends the argument in that format

**Gribouillis** · Mar-15-2022, 03:16 PM

Do you have a new logging message from the argument parser?

radioactive9 · Mar-15-2022, 03:29 PM

(Mar-15-2022, 02:20 PM)radioactive9 Wrote: Thank You

I took your Code and tried to parse my arguments the way the tool is sending and it failed. Because it is not in a expected format
import argparse
 
 
def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("-N", "--host", help="Node Name")
    parser.add_argument("-I", "--IPAddress", help="Node Name")
    parser.add_argument("-S", "--service", help="Service Name")
    parser.add_argument("-D", "--displayname", help="Display Name")
    parser.add_argument("-G", "--group", help="Group")
    parser.add_argument("-E", "--environment", help="Environement")
    parser.add_argument("-Z", "--newstate", help="Current Type of the alert")
    parser.add_argument("-A", "--appname", help="Application Name")
    parser.add_argument("-HA", "--hostappname", help="Host App Name")
    parser.add_argument("-P", "--priority", help="Priority")
    parser.add_argument("-O", "--output", help="Description")
    parser.add_argument("-T", "--eventtime", help="Event Time")
    parser.add_argument("-OS", "--os", help="Operating System")
    parser.add_argument("-OP", "--operator", help="Operator")
    parser.add_argument(
        "-NT", "--notificationtype", help="Notification Type Problem or Clear"
    )
    parser.add_argument("-ND", "--notification_default", help="Define which Operator")
    parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken
    parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")
    return parser.parse_args()
 
 
if __name__ == "__main__":
    args = get_args()
    print(args)
Quote:run Testing.py '--host', 'cccccccccc602', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-Server', '--Environment', 'PROD', '--eventtime', '2022-03-15 15:13:26 +0100', '--division', 'Tec', '--operatoros', 'GSC_W_RUN', '--operatormw', 'GSC_W_RUN', '--appname', 'TEC-Server', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99'

Obviously got error

An exception has occurred, use %tb to see the full traceback.

SystemExit: 2
Error:Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\utils\py3compat.py", line 188, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 37, in <module>
    args = get_args()

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 33, in get_args
    return parser.parse_args()

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 1733, in parse_args
    self.error(msg % ' '.join(argv))

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2389, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2376, in exit
    _sys.exit(status)

SystemExit: 2
Now i cannot change the format in which tool is sending me the data as argument. What are my options

Here I posted the errors from the code i am executing with the arguments

DeaD_EyE · (This post was last modified: Mar-15-2022, 03:39 PM by DeaD_EyE.)

I corrected some typos and added the missing arguments.

import argparse


def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("-N", "--host", help="Node Name")
    parser.add_argument("-I", "--IPAddress", help="Node Name")
    parser.add_argument("-S", "--service", help="Service Name")
    parser.add_argument("-D", "--displayname", help="Display Name")
    parser.add_argument("-G", "--group", help="Group")
    parser.add_argument("-E", "--Environment", help="Environement")
    parser.add_argument("-Z", "--newstate", help="Current Type of the alert")
    parser.add_argument("-A", "--appname", help="Application Name")
    parser.add_argument("-HA", "--hostappname", help="Host App Name")
    parser.add_argument("-P", "--priority", help="Priority")
    parser.add_argument("-O", "--output", help="Description")
    parser.add_argument("-T", "--eventtime", help="Event Time")
    parser.add_argument("-OS", "--os", help="Operating System")
    parser.add_argument("-OP", "--operator", help="Operator")
    parser.add_argument("--division")
    parser.add_argument("--operatoros")
    parser.add_argument("--operatormw")
    parser.add_argument("--Info")
    parser.add_argument(
        "-NT", "--notificationtype", help="Notification Type Problem or Clear"
    )
    parser.add_argument("-ND", "--notification_default", help="Define which Operator")
    parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken
    parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")
    return parser.parse_args()


if __name__ == "__main__":
    args = get_args()
    print(args)

Pay attention about the quoting.
The call from command line:

python t.py --host cccccccccc602 --service SN-WIN-osservices --priority WARNING --output 'WARNING: , delayed (BITS=stopped (delayed))' --os 'Windows Server 2016 Standard' --operator operator_os --notificationtype PROBLEM --notification_default operator_os --hostappname TEC-Server --Environment PROD --eventtime '2022-03-15 15:13:26 +0100' --division Tec --operatoros GSC_W_RUN --operatormw GSC_W_RUN --appname TEC-Server --Info 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts' --IPAddress 99.99.99.99

Output:
Namespace(host='cccccccccc602', IPAddress='99.99.99.99', service='SN-WIN-osservices', displayname=None, group=None, Environment='PROD', newstate=None, appname='TEC-Server', hostappname='TEC-Server', priority='WARNING', output='WARNING: , delayed (BITS=stopped (delayed))', eventtime='2022-03-15 15:13:26 +0100', os='Windows Server 2016 Standard', operator='operator_os', division='Tec', operatoros='GSC_W_RUN', operatormw='GSC_W_RUN', Info='Prueft den Status wichtiger Windows-Services auf Windows-Hosts', notificationtype='PROBLEM', notification_default='operator_os', groupoperatoros=None, groupoperatormw=None)

A trick to create a "template":
what_args.py

#!/usr/bin/env python3

import sys

print("""def get_args():
    parser = argparse.ArgumentParser()""")

parser_fmt = '    parser.add_argument("{}")'

for arg in sys.argv[1:]:
    if arg.startswith("-"):
        print(parser_fmt.format(arg))


print("""    return parser.parse_args()""")
print()

python what_args.py --host cccccccccc602 --service SN-WIN-osservices --priority WARNING --output 'WARNING: , delayed (BITS=stopped (delayed))' --os 'Windows Server 2016 Standard' --operator operator_os --notificationtype PROBLEM --notification_default operator_os --hostappname TEC-Server --Environment PROD --eventtime '2022-03-15 15:13:26 +0100' --division Tec --operatoros GSC_W_RUN --operatormw GSC_W_RUN --appname TEC-Server --Info 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts' --IPAddress 99.99.99.99

Output:def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--host")
    parser.add_argument("--service")
    parser.add_argument("--priority")
    parser.add_argument("--output")
    parser.add_argument("--os")
    parser.add_argument("--operator")
    parser.add_argument("--notificationtype")
    parser.add_argument("--notification_default")
    parser.add_argument("--hostappname")
    parser.add_argument("--Environment")
    parser.add_argument("--eventtime")
    parser.add_argument("--division")
    parser.add_argument("--operatoros")
    parser.add_argument("--operatormw")
    parser.add_argument("--appname")
    parser.add_argument("--Info")
    parser.add_argument("--IPAddress")
    return parser.parse_args()

Possibly Related Threads…
Thread		Author	Replies	Views	Last Post
	argparse io issue	pyDream	8	702	Apr-02-2024, 12:42 PM Last Post: pyDream
	Am I wrong or is Udemy wrong? String Slicing!	Mavoz	3	2,578	Nov-05-2022, 11:33 AM Last Post: Mavoz
	argparse --help in one line.	Denial	1	2,004	Sep-20-2020, 03:38 PM Last Post: deanhystad
	Argparse error when inputting values	tqader	2	2,891	Sep-11-2020, 07:42 PM Last Post: buran
	Why this pycharm warning for argparse formatter_class value?	pjfarley3	2	2,142	Sep-09-2020, 05:23 AM Last Post: pjfarley3
	Can argparse support undocumented options?	pjfarley3	3	2,225	Aug-14-2020, 06:13 AM Last Post: pjfarley3
	In ArgParse, can two compatible formatter_class values be used?	pjfarley3	2	2,601	Jul-31-2020, 02:01 PM Last Post: pjfarley3
	python gives wrong string length and wrong character	thienson30	2	3,025	Oct-15-2019, 08:54 PM Last Post: Gribouillis
	Why am I getting KeyError 'file' when using argparse?	Mike Ru	1	3,085	Jun-09-2019, 04:48 PM Last Post: metulburr
	How can I get some arguments using argparse?	Mike Ru	0	1,888	Jun-05-2019, 12:57 PM Last Post: Mike Ru

Not able to figure out what is wrong with argparse

User Panel Messages

Announcements