Python Forum

I wrote a decorator to avoid having exceptions crash my script and it's not working. Here's the wrapper:

helpers.py

def try_wrapper(func):
    """Wrapper that handles exceptions gracefully.

        :returns:       Wrapped function return value (if no error) or None (if error)
    """
    def try_wrap(*args):
        try:
            return func(*args)
            return output
        except Exception as inst:
            print(inst)
    return try_wrap

And the script so far:

validate_url.py

"""Does some basic validation of provided url

    :split_url:         Helper function using urllib.parse.urlsplit
    :validate_scheme:   Does basic validation of url scheme
"""

import urllib.parse
from helpers import try_wrapper


"""
Module constants.

    :schemes:       uri scheeas for validation
"""

schemes = ["https", "http", "ftp"]


"""Validation and helper functions.
"""

@try_wrapper
def split_url(url):
    """Splits url for other validation functions.

        :returns:       urllib.split.urlsplit object
    """
    return urllib.parse.urlsplit(url)


@try_wrapper
def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        print("Invalid url scheme {}, must be one of {}".format(split_url.scheme, schemes))


"""Testing code to be deleted later.
"""

while True:
    url = input("Enter url: ")
    split = split_url(url)
    print(split)
    split_url = validate_scheme(split)
    print(split)

And here's the output I'm getting:

Quote:Enter url: https://bla.bla.com
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
Enter url: crud://www.bla.com
SplitResult(scheme='crud', netloc='www.bla.com', path='', query='', fragment='')
Invalid url scheme crud, must be one of ['https', 'http', 'ftp']
None
Enter url: ftp
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Invalid url scheme , must be one of ['https', 'http', 'ftp']
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Enter url: ftp:/
Traceback (most recent call last):
File "validate_url.py", line 41, in <module>
split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

Why is the script crashing despite my error-checking, which is intended to print a nice error message while not crashing the script?

There's a reason why exceptions are thrown.
You should capture and analyze all of them, as it's something you're doing wrong that's creating them.

Here's something that I use (only while developing a program. once exception types are determined, you should use actual exception name)
this will display what type of exception is being thrown.
Never leave this in a finished program, it's way too broad.
you need to import sys, as it's used to identify the exception:

import sys

# ... somewhere in your code ...
    try:
        # code goes here
    except:
        print("Unexpected error:", sys.exc_info()[0])

I just read a bit about sys_exc.info in the Python Documentation. It looks I'll have to pull the decorator from the helpers library and set up a separate exceptions library. This is a bit disheartening because I'm getting horribly sidetracked in developing the url_validator library--which was itself getting sidetracked from developing a urllib.requests library--which is itself getting sidetracked from the application I want to create. I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?

My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.

(Feb-16-2018, 06:36 PM)Larz60+ Wrote: [ -> ]My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.

Logging the issues is a great idea! (Not just for this, but also because I'm learning and it would be useful to look over.)

(Feb-16-2018, 03:18 PM)league55 Wrote: [ -> ]I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?

If you write a function named validate_scheme(), it means that you are now and then expecting invalid urls. The case where an error occurs in this call can be incorporated to this invalid case.

In your situation, I would create my own exception type InvalidUrl and raise this exception

class InvalidUrl(Exception): pass

def split_url(url):
    """Splits url for other validation functions.
 
        :returns:       urllib.split.urlsplit object
    """
    try:
        return urllib.parse.urlsplit(url)
    except Exception as exc:
        raise InvalidUrl from exc

def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        raise InvalidUrl(("Invalid url scheme", split_url.scheme, "must be one of", schemes))

Now the code that uses this can catch InvalidUrl and take corrective action when an invalid url is met.

(Feb-16-2018, 06:38 AM)league55 Wrote: [ -> ]split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

That's not inside one of your decorated functions, though. validate_scheme is returning something that isn't callable (None), and then you try to call it like a function. So what you're doing is working fine, but you should also maybe have default return values, so the caller doesn't blow up when there's an error.

The problem is in the while-loop.

def split_url(url):
    pass


def validate_scheme(url):
    pass


while True:
    url = 'foo'
    split = split_url(url)
    split_url = validate_scheme(split)

It doesn't matter what validate_scheme returns.
You're assigning the name split_url, which has been assigned before to the function.
In the while loop you call this function again.

---

Here my extension to your try_wrapper.
It's one level deeper and allows to set a return_value and is using logging.

import logging
import functools

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

def try_wrapper(*, logger=None, traceback=False, retval=None):
    def func_wrapper(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            try:
                result = func(*args, **kwargs)
                return result
            except Exception as e:
                if logger and not traceback:
                    logger.debug(e)
                elif logger and traceback:
                    logger.exception(traceback)
                return retval
            else:
                return result
        return inner
    return func_wrapper


@try_wrapper()
def foo():
    return 1/0


@try_wrapper(logger=logger)
def foo_with_logger():
    return 1/0


@try_wrapper(logger=logger, retval=42)
def foo_with_logger_and_retval():
    return 1/0

print('Foo call')
ret = foo()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger')
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger in logging level debug')
logger.setLevel(logging.DEBUG)
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger_and_retval 42')
ret = foo_with_logger_and_retval()
print('Retval:', ret, 'Type:', type(ret))

Maybe you can also log tracebacks if you want. Set traceback=True