Python Forum

Full Version: Stack trace shows different exception type than print
You're currently viewing a stripped down version of our content. View the full version with proper formatting.
Python's JSON module supports deserializing Javascript infinities (although the official JSON spec doesn't). Specifically, it supports "Infinity" and "-Infinity" in particular. I need to be able to deserialize "+Infinity" though. Here is my attempt at achieving this
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except json.decoder.JSONDecodeError as e:
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            raise e
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
The result of this is that (apparently) a StopIteration exception is thrown, although its traceback looks like a JSONDecodeError
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... Got an unexpected exception from default (type: <class 'StopIteration'>) - 0 Traceback (most recent call last): File "confusion.py", line 28, in <module> print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads return cls(**kw).decode(s) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I'm baffled because the stack trace clearly shows a JSONDecodeError being raised, but at runtime when I try to catch it, it appears to have a different type. In case the class is doing something funky, I also tried catching a ValueError (its superclass) but that didn't work either.

What does work is catching a StopIteration exception, but that doesn't seem right. I'm afraid that there's something weird going on here and if I catch the wrong exception, there will be some surprise down the line. Does anyone know what's going on here?
Henlo Fren,

My naem Sheba and I luvs da slithery codes please.

The JSONDecodeError in your codes is from JSONDecoder.raw_decode which calls self.scan_once.

Here is a good codes please:
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except StopIteration as e:  # scan_once should throw a StopIteration
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            else:
                raise StopIteration(idx)
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder))
print()
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... Traceback (most recent call last): File "confusion.py", line 31, in <module> print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder)) File "/usr/lib/python3.6/json/__init__.py", line 367, in loads return cls(**kw).decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Sincerely,
Sheba Weeba Possum Woo
Thanks for the reply but I feel like I must be missing something. I already mentioned that a StopIteration error appears to work, but that's the problem. As you can see in the link you provided, raw_decode should be throwing a JSONDecodeError, not a StopIteration error, and you can see that when I print a stacktrace the type of the error appears different than if I print the type of the error object.
There are two exceptions being thrown:
  1. json.loads('+Infinity') is called.
  2. ...
  3. raw_decode('+Infinity') is called.
  4. raw_decode calls PositiveInfinityJSONDecoder._scan_once.
  5. PositiveInfinityJSONDecoder._scan_once calls self.original_scan_once (a.k.a. JSONDecoder().scan_once).
  6. self.original_scan_once raises a StopIteration as expected.
  7. PositiveInfinityJSONDecoder._scan_once prints the StopIteration exception and reraises it.
  8. raw_decode excepts StopIteration and raises JSONDecodeError.
  9. JSONDecodeError is not caught, so it prints the stack trace for JSONDecodeError.
There are two version of make_scanner in json module, c_make_scanner and py_make_scanner. We need to force Python to use Python version (py_make_scanner).
The following isn't to be an elegant solution, but it straightforward and clear:

import json
from json.decoder import PosInf, _CONSTANTS
from json.scanner import NUMBER_RE

_CONSTANTS.update({'+Infinity': PosInf})  # Update known constants 

def py_make_scanner(context):  # there are c-version, we need to force python to use version only
    parse_object = context.parse_object
    parse_array = context.parse_array
    parse_string = context.parse_string
    match_number = NUMBER_RE.match
    strict = context.strict
    parse_float = context.parse_float
    parse_int = context.parse_int
    parse_constant = context.parse_constant
    object_hook = context.object_hook
    object_pairs_hook = context.object_pairs_hook
    memo = context.memo

    def _scan_once(string, idx):
        try:
            nextchar = string[idx]
        except IndexError:
            raise StopIteration(idx) from None

        if nextchar == '"':
            return parse_string(string, idx + 1, strict)
        elif nextchar == '{':
            return parse_object((string, idx + 1), strict,
                _scan_once, object_hook, object_pairs_hook, memo)
        elif nextchar == '[':
            return parse_array((string, idx + 1), _scan_once)
        elif nextchar == 'n' and string[idx:idx + 4] == 'null':
            return None, idx + 4
        elif nextchar == 't' and string[idx:idx + 4] == 'true':
            return True, idx + 4
        elif nextchar == 'f' and string[idx:idx + 5] == 'false':
            return False, idx + 5

        m = match_number(string, idx)
        if m is not None:
            integer, frac, exp = m.groups()
            if frac or exp:
                res = parse_float(integer + (frac or '') + (exp or ''))
            else:
                res = parse_int(integer)
            return res, m.end()
        elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
            return parse_constant('NaN'), idx + 3
        elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
            return parse_constant('Infinity'), idx + 8
        elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
            return parse_constant('-Infinity'), idx + 9
        elif nextchar == '+' and string[idx:idx + 9] == '+Infinity':  #  These lines were added;
            return parse_constant('+Infinity'), idx + 9
        else:
            raise StopIteration(idx)

    def scan_once(string, idx):
        try:
            return _scan_once(string, idx)
        finally:
            memo.clear()

    return scan_once

 
class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.scan_once = py_make_scanner(self)  # Force to use python (not c) version of py_make_scnner
 

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
Thanks scidamn! I had meant to loop back around to that and forgot. There's one small improvement to what you had that I will be incorporating - if line 55 just uses "Infinity" instead of "+Infinity" then you can drop the _CONSTANTS update.