Python Forum
Problems with not having exceptions crash my script
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Problems with not having exceptions crash my script
#1
I wrote a decorator to avoid having exceptions crash my script and it's not working. Here's the wrapper:

helpers.py
def try_wrapper(func):
    """Wrapper that handles exceptions gracefully.

        :returns:       Wrapped function return value (if no error) or None (if error)
    """
    def try_wrap(*args):
        try:
            return func(*args)
            return output
        except Exception as inst:
            print(inst)
    return try_wrap
And the script so far:

validate_url.py
"""Does some basic validation of provided url

    :split_url:         Helper function using urllib.parse.urlsplit
    :validate_scheme:   Does basic validation of url scheme
"""

import urllib.parse
from helpers import try_wrapper


"""
Module constants.

    :schemes:       uri scheeas for validation
"""

schemes = ["https", "http", "ftp"]


"""Validation and helper functions.
"""

@try_wrapper
def split_url(url):
    """Splits url for other validation functions.

        :returns:       urllib.split.urlsplit object
    """
    return urllib.parse.urlsplit(url)


@try_wrapper
def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        print("Invalid url scheme {}, must be one of {}".format(split_url.scheme, schemes))


"""Testing code to be deleted later.
"""

while True:
    url = input("Enter url: ")
    split = split_url(url)
    print(split)
    split_url = validate_scheme(split)
    print(split)
And here's the output I'm getting:

Quote:Enter url: https://bla.bla.com
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
Enter url: crud://www.bla.com
SplitResult(scheme='crud', netloc='www.bla.com', path='', query='', fragment='')
Invalid url scheme crud, must be one of ['https', 'http', 'ftp']
None
Enter url: ftp
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Invalid url scheme , must be one of ['https', 'http', 'ftp']
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Enter url: ftp:/
Traceback (most recent call last):
File "validate_url.py", line 41, in <module>
split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

Why is the script crashing despite my error-checking, which is intended to print a nice error message while not crashing the script?
Reply
#2
There's a reason why exceptions are thrown.
You should capture and analyze all of them, as it's something you're doing wrong that's creating them.

Here's something that I use (only while developing a program. once exception types are determined, you should use actual exception name)
this will display what type of exception is being thrown.
Never leave this in a finished program, it's way too broad.
you need to import sys, as it's used to identify the exception:
import sys

# ... somewhere in your code ...
    try:
        # code goes here
    except:
        print("Unexpected error:", sys.exc_info()[0])
Reply
#3
I just read a bit about sys_exc.info in the Python Documentation. It looks I'll have to pull the decorator from the helpers library and set up a separate exceptions library. This is a bit disheartening because I'm getting horribly sidetracked in developing the url_validator library--which was itself getting sidetracked from developing a urllib.requests library--which is itself getting sidetracked from the application I want to create. I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?
Reply
#4
My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.
Reply
#5
(Feb-16-2018, 06:36 PM)Larz60+ Wrote: My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.

Logging the issues is a great idea! (Not just for this, but also because I'm learning and it would be useful to look over.)
Reply
#6
(Feb-16-2018, 03:18 PM)league55 Wrote: I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?
If you write a function named validate_scheme(), it means that you are now and then expecting invalid urls. The case where an error occurs in this call can be incorporated to this invalid case.

In your situation, I would create my own exception type InvalidUrl and raise this exception
class InvalidUrl(Exception): pass

def split_url(url):
    """Splits url for other validation functions.
 
        :returns:       urllib.split.urlsplit object
    """
    try:
        return urllib.parse.urlsplit(url)
    except Exception as exc:
        raise InvalidUrl from exc

def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        raise InvalidUrl(("Invalid url scheme", split_url.scheme, "must be one of", schemes))
Now the code that uses this can catch InvalidUrl and take corrective action when an invalid url is met.
Reply
#7
(Feb-16-2018, 06:38 AM)league55 Wrote: split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

That's not inside one of your decorated functions, though. validate_scheme is returning something that isn't callable (None), and then you try to call it like a function. So what you're doing is working fine, but you should also maybe have default return values, so the caller doesn't blow up when there's an error.
Reply
#8
The problem is in the while-loop.

def split_url(url):
    pass


def validate_scheme(url):
    pass


while True:
    url = 'foo'
    split = split_url(url)
    split_url = validate_scheme(split)
It doesn't matter what validate_scheme returns.
You're assigning the name split_url, which has been assigned before to the function.
In the while loop you call this function again.

Here my extension to your try_wrapper.
It's one level deeper and allows to set a return_value and is using logging.

import logging
import functools

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

def try_wrapper(*, logger=None, traceback=False, retval=None):
    def func_wrapper(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            try:
                result = func(*args, **kwargs)
                return result
            except Exception as e:
                if logger and not traceback:
                    logger.debug(e)
                elif logger and traceback:
                    logger.exception(traceback)
                return retval
            else:
                return result
        return inner
    return func_wrapper


@try_wrapper()
def foo():
    return 1/0


@try_wrapper(logger=logger)
def foo_with_logger():
    return 1/0


@try_wrapper(logger=logger, retval=42)
def foo_with_logger_and_retval():
    return 1/0

print('Foo call')
ret = foo()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger')
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger in logging level debug')
logger.setLevel(logging.DEBUG)
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger_and_retval 42')
ret = foo_with_logger_and_retval()
print('Retval:', ret, 'Type:', type(ret))
Maybe you can also log tracebacks if you want. Set traceback=True
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  PiCamera - print exceptions? korenron 2 792 Dec-15-2022, 10:48 PM
Last Post: Larz60+
  Class exceptions DPaul 1 1,259 Mar-11-2022, 09:01 AM
Last Post: Gribouillis
  Catching a crash within a library code ebolisa 9 3,067 Nov-22-2021, 11:02 AM
Last Post: bowlofred
  Python Crash Course ( 2nd edition) alok 1 1,844 Jul-19-2021, 05:55 PM
Last Post: snippsat
  is this a good way to catch exceptions? korenron 14 4,593 Jul-05-2021, 06:20 PM
Last Post: hussaind
  Python, exceptions KingKhan248 6 2,945 Nov-15-2020, 06:54 AM
Last Post: buran
  Split string between two different delimiters, with exceptions DreamingInsanity 2 1,979 Aug-24-2020, 08:23 AM
Last Post: DreamingInsanity
  handling 2 exceptions at once Skaperen 2 2,262 Jun-27-2020, 08:55 AM
Last Post: Yoriz
  remove spaces with exceptions catosp 4 2,363 May-29-2020, 09:32 AM
Last Post: catosp
  Looking for advice and Guidance on Exceptions used within Functions paul41 1 2,108 Nov-14-2019, 12:33 AM
Last Post: Larz60+

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020