Not able to figure out what is wrong with argparse

[radioactive9]

Hello

I am facing a very unique issue. I have written the following code which will be called by a Tool and pass certain arguments. I was using argpass thinking it will be easy. But I am not able to figure out what is going wrong with it

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39

if __name__ == "__main__":     log_path = "/var/log/tooly2/"     log_file = "notifications.log"     log_file_path = log_path + log_file     if os.access(log_path, os.W_OK) and os.access(log_path + log_file, os.W_OK):         # Initialize the default/basic and a custom logger         logging.basicConfig(filename=log_file_path, level=logging.DEBUG, format='%(process)d %(asctime)s %(levelname)s - %(funcName)s: %(message)s', datefmt="%Y-%m-%d %H:%M:%S")     else:         print ("The log directory or the log file is not writable for this script, so no logging is possible. Exiting now.")         sys.exit(1)     try:         n = len(sys.argv)         logging.debug('Total arguments passed: %s', n) #check the arguments         logging.debug('I can print the first argument %s', sys.argv[1]) #check arguments 1         logging.debug('I can print the first argument %s', sys.argv[2]) #check argument 2         parser = argparse.ArgumentParser()         parser.add_argument("-N", "--host", help="Node Name") #taken         parser.add_argument("-I", "--IPAddress", help="Node Name") #taken         parser.add_argument("-S", "--service", help="Service Name")         parser.add_argument("-D", "--displayname", help="Display Name")         parser.add_argument("-G", "--group", help="Group")         parser.add_argument("-E", "--environment", help="Environement") #Taken         parser.add_argument("-Z", "--newstate", help="Current Type of the alert")         parser.add_argument("-A", "--appname", help="Application Name")         parser.add_argument("-HA", "--hostappname", help="Host App Name") #taken         parser.add_argument("-P", "--priority", help="Priority") #taken         parser.add_argument("-O", "--output", help="Description") #taken         parser.add_argument("-T", "--eventtime", help="Event Time") #taken         parser.add_argument("-OS", "--os", help="Operating System") #taken         parser.add_argument("-OP", "--operator", help="Operator") #taken         parser.add_argument("-NT", "--notificationtype", help="Notification Type Problem or Clear") #taken         parser.add_argument("-ND", "--notification_default", help="Define which Operator") #taken         parser.add_argument("-NM", "--groupoperatoros", help="OS Group") #taken         parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group") #taken         arguments = parser.parse_args()         logging.debug('I am not able to reach here')     except Exception as e:         logging.debug('This message should go to the log file %s', e )         exit(1)

In the logs I am getting

Output:
5618 2022-03-15 12:36:50 DEBUG - <module>: Total arguments passed: 35
5618 2022-03-15 12:36:50 DEBUG - <module>: I can print the first argument key --host
5618 2022-03-15 12:36:50 DEBUG - <module>: I can print the second argument value ccccccccccc85
5629 2022-03-15 12:37:10 DEBUG - <module>: Total arguments passed: 35
5629 2022-03-15 12:37:10 DEBUG - <module>: I can print the first argument key --host
5629 2022-03-15 12:37:10 DEBUG - <module>: I can print the second argument value xxxxxxxxxx190
5633 2022-03-15 12:37:26 DEBUG - <module>: Total arguments passed: 35

So as you see the line
logging.debug('I am not able to reach here') is not printing which means the code is getting lost somewhere in the argpass

Not sure how to even troubleshoot and figure out the issue

Any direction will be very helpful

[Gribouillis]

This code is incomplete, I was unable to reproduce the error.

[radioactive9]

This code is incomplete, I was unable to reproduce the error.

In the code line 36 doesn't get executed. It doesn't throw any error as well. That is my problem. I do not see any reason why that line will not execute and stop there. There are lot of functions that will be called after that. But for now my problem is it doesn't execute that line 36 and stops there. Doesn't throw error as well

Its not a compilation error as well - as I am getting the output in Logging for line 13 14 15

I intentionally put the try catch block for argparse as I was not able to figure out the problem.

EDIT:

To do further troubleshooting I added the following line in the code

1	logging.debug('Complete list of argument received %s', sys.argv[1:])

I got the arguments like this. Is this format OK for argparse to parse properly

20205 2022-03-15 14:30:15 DEBUG - <module>: Complete list of argument received ['--host', 'ccccccccc87', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-User_Role', '--Environment', 'PROD', '--eventtime', '2022-03-15 14:30:14 +0100', '--division', 'Tec', '--operatoros', 'G_WNT_RUN', '--operatormw', 'G_NT_RUN', '--appname', 'TEC-User_Role', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99']

[DeaD_EyE]

Your problem is your debugging code.
The debugging code had a bug. If no arguments are supplied, the index access sys.argv[1] will throw the exception.
Catching all Exceptions with Exception is wrong. Instead, let fail your code in the first place.
Then look which Exceptions could occur and which Exceptions you expect to catch.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38

import argparse     def get_args():     parser = argparse.ArgumentParser()     parser.add_argument("-N", "--host", help="Node Name")     parser.add_argument("-I", "--IPAddress", help="Node Name")     parser.add_argument("-S", "--service", help="Service Name")     parser.add_argument("-D", "--displayname", help="Display Name")     parser.add_argument("-G", "--group", help="Group")     parser.add_argument("-E", "--environment", help="Environement")     parser.add_argument("-Z", "--newstate", help="Current Type of the alert")     parser.add_argument("-A", "--appname", help="Application Name")     parser.add_argument("-HA", "--hostappname", help="Host App Name")     parser.add_argument("-P", "--priority", help="Priority")     parser.add_argument("-O", "--output", help="Description")     parser.add_argument("-T", "--eventtime", help="Event Time")     parser.add_argument("-OS", "--os", help="Operating System")     parser.add_argument("-OP", "--operator", help="Operator")     parser.add_argument(         "-NT", "--notificationtype", help="Notification Type Problem or Clear"     )     parser.add_argument("-ND", "--notification_default", help="Define which Operator")     parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken     parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")     return parser.parse_args()     if __name__ == "__main__":     args = get_args()     print(args)     # This error came from line 14 # Line 14: logging.debug('I can print the first argument %s', sys.argv[1]) #check arguments 1 # 14927 2022-03-15 14:18:52 DEBUG - <module>: This message should go to the log file list index out of range   # conclusion, your debugging code introduced new bugs

If you want to use short and long arguments which are required, then you can add required=True.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

import argparse     parser = argparse.ArgumentParser() parser.add_argument("-o", "--output", required=True)   # simulating the commandline args = parser.parse_args(["-o", "test_OUTPUT"]) print(args)     # This here will raise an SystemExit exception because of wrong input # usually the program will then exit with the supplied code try:     parser.parse_args([]) except SystemExit as e:     print("Method parse_args wanted to exit the programm with return_code", e.code)

[radioactive9]

Thank You

I took your Code and tried to parse my arguments the way the tool is sending and it failed. Because it is not in a expected format

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

import argparse       def get_args():     parser = argparse.ArgumentParser()     parser.add_argument("-N", "--host", help="Node Name")     parser.add_argument("-I", "--IPAddress", help="Node Name")     parser.add_argument("-S", "--service", help="Service Name")     parser.add_argument("-D", "--displayname", help="Display Name")     parser.add_argument("-G", "--group", help="Group")     parser.add_argument("-E", "--environment", help="Environement")     parser.add_argument("-Z", "--newstate", help="Current Type of the alert")     parser.add_argument("-A", "--appname", help="Application Name")     parser.add_argument("-HA", "--hostappname", help="Host App Name")     parser.add_argument("-P", "--priority", help="Priority")     parser.add_argument("-O", "--output", help="Description")     parser.add_argument("-T", "--eventtime", help="Event Time")     parser.add_argument("-OS", "--os", help="Operating System")     parser.add_argument("-OP", "--operator", help="Operator")     parser.add_argument(         "-NT", "--notificationtype", help="Notification Type Problem or Clear"     )     parser.add_argument("-ND", "--notification_default", help="Define which Operator")     parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken     parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")     return parser.parse_args()       if __name__ == "__main__":     args = get_args()     print(args)

run Testing.py '--host', 'cccccccccc602', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-Server', '--Environment', 'PROD', '--eventtime', '2022-03-15 15:13:26 +0100', '--division', 'Tec', '--operatoros', 'GSC_W_RUN', '--operatormw', 'GSC_W_RUN', '--appname', 'TEC-Server', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99'

Obviously got error

An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

Error:
Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\utils\py3compat.py", line 188, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 37, in <module>
    args = get_args()

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 33, in get_args
    return parser.parse_args()

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 1733, in parse_args
    self.error(msg % ' '.join(argv))

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2389, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2376, in exit
    _sys.exit(status)

SystemExit: 2

Now i cannot change the format in which tool is sending me the data as argument. What are my options

[Gribouillis]

Try this hack to log the error

1 2 3 4 5 6 7 8 9 10 11 12

parser = argparse.ArgumentParser() ...   _error = parser.error   def error(message):     logging.debug('argparse parser error %s', message)     _error(message)   parser.error = error       arguments = parser.parse_args()

[radioactive9]

Try this hack to log the error

1 2 3 4 5 6 7 8 9 10 11 12

parser = argparse.ArgumentParser() ...   _error = parser.error   def error(message):     logging.debug('argparse parser error %s', message)     _error(message)   parser.error = error       arguments = parser.parse_args()

Hi

Do you think my problem is the argument's that is being passed ? It is not in expected format. Problem is tool sends the argument in that format

[Gribouillis]

Do you have a new logging message from the argument parser?

[radioactive9]

Thank You

I took your Code and tried to parse my arguments the way the tool is sending and it failed. Because it is not in a expected format

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31

import argparse       def get_args():     parser = argparse.ArgumentParser()     parser.add_argument("-N", "--host", help="Node Name")     parser.add_argument("-I", "--IPAddress", help="Node Name")     parser.add_argument("-S", "--service", help="Service Name")     parser.add_argument("-D", "--displayname", help="Display Name")     parser.add_argument("-G", "--group", help="Group")     parser.add_argument("-E", "--environment", help="Environement")     parser.add_argument("-Z", "--newstate", help="Current Type of the alert")     parser.add_argument("-A", "--appname", help="Application Name")     parser.add_argument("-HA", "--hostappname", help="Host App Name")     parser.add_argument("-P", "--priority", help="Priority")     parser.add_argument("-O", "--output", help="Description")     parser.add_argument("-T", "--eventtime", help="Event Time")     parser.add_argument("-OS", "--os", help="Operating System")     parser.add_argument("-OP", "--operator", help="Operator")     parser.add_argument(         "-NT", "--notificationtype", help="Notification Type Problem or Clear"     )     parser.add_argument("-ND", "--notification_default", help="Define which Operator")     parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken     parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")     return parser.parse_args()       if __name__ == "__main__":     args = get_args()     print(args)

run Testing.py '--host', 'cccccccccc602', '--service', 'SN-WIN-osservices', '--priority', 'WARNING', '--output', 'WARNING: , delayed (BITS=stopped (delayed))', '--os', 'Windows Server 2016 Standard', '--operator', 'operator_os', '--notificationtype', 'PROBLEM', '--notification_default', 'operator_os', '--hostappname', 'TEC-Server', '--Environment', 'PROD', '--eventtime', '2022-03-15 15:13:26 +0100', '--division', 'Tec', '--operatoros', 'GSC_W_RUN', '--operatormw', 'GSC_W_RUN', '--appname', 'TEC-Server', '--Info', 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts', '--IPAddress', '99.99.99.99'

Obviously got error

An exception has occurred, use %tb to see the full traceback.

SystemExit: 2

Error:
Traceback (most recent call last):

  File "C:\ProgramData\Anaconda3\lib\site-packages\IPython\utils\py3compat.py", line 188, in execfile
    exec(compiler(f.read(), fname, 'exec'), glob, loc)

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 37, in <module>
    args = get_args()

  File "D:\Scripts\ServiceNowIntegration\Testing.py", line 33, in get_args
    return parser.parse_args()

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 1733, in parse_args
    self.error(msg % ' '.join(argv))

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2389, in error
    self.exit(2, _('%(prog)s: error: %(message)s\n') % args)

  File "C:\ProgramData\Anaconda3\lib\argparse.py", line 2376, in exit
    _sys.exit(status)

SystemExit: 2

Now i cannot change the format in which tool is sending me the data as argument. What are my options

Here I posted the errors from the code i am executing with the arguments

[DeaD_EyE]

I corrected some typos and added the missing arguments.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35

import argparse     def get_args():     parser = argparse.ArgumentParser()     parser.add_argument("-N", "--host", help="Node Name")     parser.add_argument("-I", "--IPAddress", help="Node Name")     parser.add_argument("-S", "--service", help="Service Name")     parser.add_argument("-D", "--displayname", help="Display Name")     parser.add_argument("-G", "--group", help="Group")     parser.add_argument("-E", "--Environment", help="Environement")     parser.add_argument("-Z", "--newstate", help="Current Type of the alert")     parser.add_argument("-A", "--appname", help="Application Name")     parser.add_argument("-HA", "--hostappname", help="Host App Name")     parser.add_argument("-P", "--priority", help="Priority")     parser.add_argument("-O", "--output", help="Description")     parser.add_argument("-T", "--eventtime", help="Event Time")     parser.add_argument("-OS", "--os", help="Operating System")     parser.add_argument("-OP", "--operator", help="Operator")     parser.add_argument("--division")     parser.add_argument("--operatoros")     parser.add_argument("--operatormw")     parser.add_argument("--Info")     parser.add_argument(         "-NT", "--notificationtype", help="Notification Type Problem or Clear"     )     parser.add_argument("-ND", "--notification_default", help="Define which Operator")     parser.add_argument("-NM", "--groupoperatoros", help="OS Group")  # taken     parser.add_argument("-NO", "--groupoperatormw", help="Middleware Group")     return parser.parse_args()     if __name__ == "__main__":     args = get_args()     print(args)

Pay attention about the quoting.
The call from command line:

1

python t.py --host cccccccccc602 --service SN-WIN-osservices --priority WARNING --output 'WARNING: , delayed (BITS=stopped (delayed))' --os 'Windows Server 2016 Standard' --operator operator_os --notificationtype PROBLEM --notification_default operator_os --hostappname TEC-Server --Environment PROD --eventtime '2022-03-15 15:13:26 +0100' --division Tec --operatoros GSC_W_RUN --operatormw GSC_W_RUN --appname TEC-Server --Info 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts' --IPAddress 99.99.99.99

Output:
Namespace(host='cccccccccc602', IPAddress='99.99.99.99', service='SN-WIN-osservices', displayname=None, group=None, Environment='PROD', newstate=None, appname='TEC-Server', hostappname='TEC-Server', priority='WARNING', output='WARNING: , delayed (BITS=stopped (delayed))', eventtime='2022-03-15 15:13:26 +0100', os='Windows Server 2016 Standard', operator='operator_os', division='Tec', operatoros='GSC_W_RUN', operatormw='GSC_W_RUN', Info='Prueft den Status wichtiger Windows-Services auf Windows-Hosts', notificationtype='PROBLEM', notification_default='operator_os', groupoperatoros=None, groupoperatormw=None)

A trick to create a "template":
what_args.py

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

#!/usr/bin/env python3   import sys   print("""def get_args():     parser = argparse.ArgumentParser()""")   parser_fmt = '    parser.add_argument("{}")'   for arg in sys.argv[1:]:     if arg.startswith("-"):         print(parser_fmt.format(arg))     print("""    return parser.parse_args()""") print()

1

python what_args.py --host cccccccccc602 --service SN-WIN-osservices --priority WARNING --output 'WARNING: , delayed (BITS=stopped (delayed))' --os 'Windows Server 2016 Standard' --operator operator_os --notificationtype PROBLEM --notification_default operator_os --hostappname TEC-Server --Environment PROD --eventtime '2022-03-15 15:13:26 +0100' --division Tec --operatoros GSC_W_RUN --operatormw GSC_W_RUN --appname TEC-Server --Info 'Prueft den Status wichtiger Windows-Services auf Windows-Hosts' --IPAddress 99.99.99.99

Output:
def get_args():
    parser = argparse.ArgumentParser()
    parser.add_argument("--host")
    parser.add_argument("--service")
    parser.add_argument("--priority")
    parser.add_argument("--output")
    parser.add_argument("--os")
    parser.add_argument("--operator")
    parser.add_argument("--notificationtype")
    parser.add_argument("--notification_default")
    parser.add_argument("--hostappname")
    parser.add_argument("--Environment")
    parser.add_argument("--eventtime")
    parser.add_argument("--division")
    parser.add_argument("--operatoros")
    parser.add_argument("--operatormw")
    parser.add_argument("--appname")
    parser.add_argument("--Info")
    parser.add_argument("--IPAddress")
    return parser.parse_args()