Python Forum
Stack trace shows different exception type than print - Printable Version

+- Python Forum (https://python-forum.io)
+-- Forum: Python Coding (https://python-forum.io/forum-7.html)
+--- Forum: General Coding Help (https://python-forum.io/forum-8.html)
+--- Thread: Stack trace shows different exception type than print (/thread-17150.html)



Stack trace shows different exception type than print - micseydel - Mar-30-2019

Python's JSON module supports deserializing Javascript infinities (although the official JSON spec doesn't). Specifically, it supports "Infinity" and "-Infinity" in particular. I need to be able to deserialize "+Infinity" though. Here is my attempt at achieving this
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except json.decoder.JSONDecodeError as e:
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            raise e
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
The result of this is that (apparently) a StopIteration exception is thrown, although its traceback looks like a JSONDecodeError
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... Got an unexpected exception from default (type: <class 'StopIteration'>) - 0 Traceback (most recent call last): File "confusion.py", line 28, in <module> print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads return cls(**kw).decode(s) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I'm baffled because the stack trace clearly shows a JSONDecodeError being raised, but at runtime when I try to catch it, it appears to have a different type. In case the class is doing something funky, I also tried catching a ValueError (its superclass) but that didn't work either.

What does work is catching a StopIteration exception, but that doesn't seem right. I'm afraid that there's something weird going on here and if I catch the wrong exception, there will be some surprise down the line. Does anyone know what's going on here?


RE: Stack trace shows different exception type than print - sheeba_weeba - Mar-31-2019

Henlo Fren,

My naem Sheba and I luvs da slithery codes please.

The JSONDecodeError in your codes is from JSONDecoder.raw_decode which calls self.scan_once.

Here is a good codes please:
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except StopIteration as e:  # scan_once should throw a StopIteration
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            else:
                raise StopIteration(idx)
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder))
print()
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... Traceback (most recent call last): File "confusion.py", line 31, in <module> print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder)) File "/usr/lib/python3.6/json/__init__.py", line 367, in loads return cls(**kw).decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Sincerely,
Sheba Weeba Possum Woo


RE: Stack trace shows different exception type than print - micseydel - Mar-31-2019

Thanks for the reply but I feel like I must be missing something. I already mentioned that a StopIteration error appears to work, but that's the problem. As you can see in the link you provided, raw_decode should be throwing a JSONDecodeError, not a StopIteration error, and you can see that when I print a stacktrace the type of the error appears different than if I print the type of the error object.


RE: Stack trace shows different exception type than print - sheeba_weeba - Mar-31-2019

There are two exceptions being thrown:
  1. json.loads('+Infinity') is called.
  2. ...
  3. raw_decode('+Infinity') is called.
  4. raw_decode calls PositiveInfinityJSONDecoder._scan_once.
  5. PositiveInfinityJSONDecoder._scan_once calls self.original_scan_once (a.k.a. JSONDecoder().scan_once).
  6. self.original_scan_once raises a StopIteration as expected.
  7. PositiveInfinityJSONDecoder._scan_once prints the StopIteration exception and reraises it.
  8. raw_decode excepts StopIteration and raises JSONDecodeError.
  9. JSONDecodeError is not caught, so it prints the stack trace for JSONDecodeError.



RE: Stack trace shows different exception type than print - scidam - Apr-01-2019

There are two version of make_scanner in json module, c_make_scanner and py_make_scanner. We need to force Python to use Python version (py_make_scanner).
The following isn't to be an elegant solution, but it straightforward and clear:

import json
from json.decoder import PosInf, _CONSTANTS
from json.scanner import NUMBER_RE

_CONSTANTS.update({'+Infinity': PosInf})  # Update known constants 

def py_make_scanner(context):  # there are c-version, we need to force python to use version only
    parse_object = context.parse_object
    parse_array = context.parse_array
    parse_string = context.parse_string
    match_number = NUMBER_RE.match
    strict = context.strict
    parse_float = context.parse_float
    parse_int = context.parse_int
    parse_constant = context.parse_constant
    object_hook = context.object_hook
    object_pairs_hook = context.object_pairs_hook
    memo = context.memo

    def _scan_once(string, idx):
        try:
            nextchar = string[idx]
        except IndexError:
            raise StopIteration(idx) from None

        if nextchar == '"':
            return parse_string(string, idx + 1, strict)
        elif nextchar == '{':
            return parse_object((string, idx + 1), strict,
                _scan_once, object_hook, object_pairs_hook, memo)
        elif nextchar == '[':
            return parse_array((string, idx + 1), _scan_once)
        elif nextchar == 'n' and string[idx:idx + 4] == 'null':
            return None, idx + 4
        elif nextchar == 't' and string[idx:idx + 4] == 'true':
            return True, idx + 4
        elif nextchar == 'f' and string[idx:idx + 5] == 'false':
            return False, idx + 5

        m = match_number(string, idx)
        if m is not None:
            integer, frac, exp = m.groups()
            if frac or exp:
                res = parse_float(integer + (frac or '') + (exp or ''))
            else:
                res = parse_int(integer)
            return res, m.end()
        elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
            return parse_constant('NaN'), idx + 3
        elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
            return parse_constant('Infinity'), idx + 8
        elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
            return parse_constant('-Infinity'), idx + 9
        elif nextchar == '+' and string[idx:idx + 9] == '+Infinity':  #  These lines were added;
            return parse_constant('+Infinity'), idx + 9
        else:
            raise StopIteration(idx)

    def scan_once(string, idx):
        try:
            return _scan_once(string, idx)
        finally:
            memo.clear()

    return scan_once

 
class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.scan_once = py_make_scanner(self)  # Force to use python (not c) version of py_make_scnner
 

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))



RE: Stack trace shows different exception type than print - micseydel - Apr-01-2019

Thanks scidamn! I had meant to loop back around to that and forgot. There's one small improvement to what you had that I will be incorporating - if line 55 just uses "Infinity" instead of "+Infinity" then you can drop the _CONSTANTS update.