Python Forum
Problems with not having exceptions crash my script - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Problems with not having exceptions crash my script (/thread-8350.html)

Problems with not having exceptions crash my script - league55 - Feb-16-2018

I wrote a decorator to avoid having exceptions crash my script and it's not working. Here's the wrapper:

helpers.py
def try_wrapper(func):
    """Wrapper that handles exceptions gracefully.

        :returns:       Wrapped function return value (if no error) or None (if error)
    """
    def try_wrap(*args):
        try:
            return func(*args)
            return output
        except Exception as inst:
            print(inst)
    return try_wrap
And the script so far:

validate_url.py
"""Does some basic validation of provided url

    :split_url:         Helper function using urllib.parse.urlsplit
    :validate_scheme:   Does basic validation of url scheme
"""

import urllib.parse
from helpers import try_wrapper


"""
Module constants.

    :schemes:       uri scheeas for validation
"""

schemes = ["https", "http", "ftp"]


"""Validation and helper functions.
"""

@try_wrapper
def split_url(url):
    """Splits url for other validation functions.

        :returns:       urllib.split.urlsplit object
    """
    return urllib.parse.urlsplit(url)


@try_wrapper
def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        print("Invalid url scheme {}, must be one of {}".format(split_url.scheme, schemes))


"""Testing code to be deleted later.
"""

while True:
    url = input("Enter url: ")
    split = split_url(url)
    print(split)
    split_url = validate_scheme(split)
    print(split)
And here's the output I'm getting:

Quote:Enter url: https://bla.bla.com
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
SplitResult(scheme='https', netloc='bla.bla.com', path='', query='', fragment='')
Enter url: crud://www.bla.com
SplitResult(scheme='crud', netloc='www.bla.com', path='', query='', fragment='')
Invalid url scheme crud, must be one of ['https', 'http', 'ftp']
None
Enter url: ftp
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Invalid url scheme , must be one of ['https', 'http', 'ftp']
SplitResult(scheme='', netloc='', path='ftp', query='', fragment='')
Enter url: ftp:/
Traceback (most recent call last):
File "validate_url.py", line 41, in <module>
split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

Why is the script crashing despite my error-checking, which is intended to print a nice error message while not crashing the script?

RE: Problems with not having exceptions crash my script - Larz60+ - Feb-16-2018

There's a reason why exceptions are thrown.
You should capture and analyze all of them, as it's something you're doing wrong that's creating them.

Here's something that I use (only while developing a program. once exception types are determined, you should use actual exception name)
this will display what type of exception is being thrown.
Never leave this in a finished program, it's way too broad.
you need to import sys, as it's used to identify the exception:
import sys

# ... somewhere in your code ...
    try:
        # code goes here
    except:
        print("Unexpected error:", sys.exc_info()[0])


RE: Problems with not having exceptions crash my script - league55 - Feb-16-2018

I just read a bit about sys_exc.info in the Python Documentation. It looks I'll have to pull the decorator from the helpers library and set up a separate exceptions library. This is a bit disheartening because I'm getting horribly sidetracked in developing the url_validator library--which was itself getting sidetracked from developing a urllib.requests library--which is itself getting sidetracked from the application I want to create. I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?

RE: Problems with not having exceptions crash my script - Larz60+ - Feb-16-2018

My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.

RE: Problems with not having exceptions crash my script - SeabassG33 - Feb-16-2018

(Feb-16-2018, 06:36 PM)Larz60+ Wrote: My personal experience is to fix each and every exception that I cause,
and to acknowledge those that are caused by bad data (or other things that I can't fix)
, and log them for review later. This way the program doesn't cause a crash, but I can
review the log and decide what to do later.

Not fixing problems as encountered will sooner or later come back to bite you. It's
a lot easier to take care of the alligators when there not in the company of other beasts.

Logging the issues is a great idea! (Not just for this, but also because I'm learning and it would be useful to look over.)

RE: Problems with not having exceptions crash my script - Gribouillis - Feb-16-2018

(Feb-16-2018, 03:18 PM)league55 Wrote: I'm not sure whether I should shelve exception handling until I've got the more important and interesting parts of the application complete. What do you think?
If you write a function named validate_scheme(), it means that you are now and then expecting invalid urls. The case where an error occurs in this call can be incorporated to this invalid case.

In your situation, I would create my own exception type InvalidUrl and raise this exception
class InvalidUrl(Exception): pass

def split_url(url):
    """Splits url for other validation functions.
 
        :returns:       urllib.split.urlsplit object
    """
    try:
        return urllib.parse.urlsplit(url)
    except Exception as exc:
        raise InvalidUrl from exc

def validate_scheme(split_url):
    """Validates schema of url against a limited list of valid schemas.
    """
    if split_url.scheme in schemes:
        return split_url
    else:
        raise InvalidUrl(("Invalid url scheme", split_url.scheme, "must be one of", schemes))
Now the code that uses this can catch InvalidUrl and take corrective action when an invalid url is met.

RE: Problems with not having exceptions crash my script - nilamo - Feb-16-2018

(Feb-16-2018, 06:38 AM)league55 Wrote: split = split_url(url)
TypeError: 'NoneType' object is not callable
<script crashes here>

That's not inside one of your decorated functions, though. validate_scheme is returning something that isn't callable (None), and then you try to call it like a function. So what you're doing is working fine, but you should also maybe have default return values, so the caller doesn't blow up when there's an error.

RE: Problems with not having exceptions crash my script - DeaD_EyE - Feb-16-2018

The problem is in the while-loop.

def split_url(url):
    pass


def validate_scheme(url):
    pass


while True:
    url = 'foo'
    split = split_url(url)
    split_url = validate_scheme(split)
It doesn't matter what validate_scheme returns.
You're assigning the name split_url, which has been assigned before to the function.
In the while loop you call this function again.
Here my extension to your try_wrapper.
It's one level deeper and allows to set a return_value and is using logging.

import logging
import functools

logger = logging.getLogger(__name__)
logger.setLevel(logging.INFO)

def try_wrapper(*, logger=None, traceback=False, retval=None):
    def func_wrapper(func):
        @functools.wraps(func)
        def inner(*args, **kwargs):
            try:
                result = func(*args, **kwargs)
                return result
            except Exception as e:
                if logger and not traceback:
                    logger.debug(e)
                elif logger and traceback:
                    logger.exception(traceback)
                return retval
            else:
                return result
        return inner
    return func_wrapper


@try_wrapper()
def foo():
    return 1/0


@try_wrapper(logger=logger)
def foo_with_logger():
    return 1/0


@try_wrapper(logger=logger, retval=42)
def foo_with_logger_and_retval():
    return 1/0

print('Foo call')
ret = foo()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger')
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger in logging level debug')
logger.setLevel(logging.DEBUG)
ret = foo_with_logger()
print('Retval:', ret, 'Type:', type(ret))
print()
print('foo_with_logger_and_retval 42')
ret = foo_with_logger_and_retval()
print('Retval:', ret, 'Type:', type(ret))
Maybe you can also log tracebacks if you want. Set traceback=True