Python Forum
Stack trace shows different exception type than print
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Stack trace shows different exception type than print
#1
Python's JSON module supports deserializing Javascript infinities (although the official JSON spec doesn't). Specifically, it supports "Infinity" and "-Infinity" in particular. I need to be able to deserialize "+Infinity" though. Here is my attempt at achieving this
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except json.decoder.JSONDecodeError as e:
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            raise e
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
The result of this is that (apparently) a StopIteration exception is thrown, although its traceback looks like a JSONDecodeError
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... Got an unexpected exception from default (type: <class 'StopIteration'>) - 0 Traceback (most recent call last): File "confusion.py", line 28, in <module> print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads return cls(**kw).decode(s) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I'm baffled because the stack trace clearly shows a JSONDecodeError being raised, but at runtime when I try to catch it, it appears to have a different type. In case the class is doing something funky, I also tried catching a ValueError (its superclass) but that didn't work either.

What does work is catching a StopIteration exception, but that doesn't seem right. I'm afraid that there's something weird going on here and if I catch the wrong exception, there will be some surprise down the line. Does anyone know what's going on here?
Reply
#2
Henlo Fren,

My naem Sheba and I luvs da slithery codes please.

The JSONDecodeError in your codes is from JSONDecoder.raw_decode which calls self.scan_once.

Here is a good codes please:
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except StopIteration as e:  # scan_once should throw a StopIteration
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            else:
                raise StopIteration(idx)
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder))
print()
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... Traceback (most recent call last): File "confusion.py", line 31, in <module> print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder)) File "/usr/lib/python3.6/json/__init__.py", line 367, in loads return cls(**kw).decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Sincerely,
Sheba Weeba Possum Woo
Reply
#3
Thanks for the reply but I feel like I must be missing something. I already mentioned that a StopIteration error appears to work, but that's the problem. As you can see in the link you provided, raw_decode should be throwing a JSONDecodeError, not a StopIteration error, and you can see that when I print a stacktrace the type of the error appears different than if I print the type of the error object.
Reply
#4
There are two exceptions being thrown:
  1. json.loads('+Infinity') is called.
  2. ...
  3. raw_decode('+Infinity') is called.
  4. raw_decode calls PositiveInfinityJSONDecoder._scan_once.
  5. PositiveInfinityJSONDecoder._scan_once calls self.original_scan_once (a.k.a. JSONDecoder().scan_once).
  6. self.original_scan_once raises a StopIteration as expected.
  7. PositiveInfinityJSONDecoder._scan_once prints the StopIteration exception and reraises it.
  8. raw_decode excepts StopIteration and raises JSONDecodeError.
  9. JSONDecodeError is not caught, so it prints the stack trace for JSONDecodeError.
Reply
#5
There are two version of make_scanner in json module, c_make_scanner and py_make_scanner. We need to force Python to use Python version (py_make_scanner).
The following isn't to be an elegant solution, but it straightforward and clear:

import json
from json.decoder import PosInf, _CONSTANTS
from json.scanner import NUMBER_RE

_CONSTANTS.update({'+Infinity': PosInf})  # Update known constants 

def py_make_scanner(context):  # there are c-version, we need to force python to use version only
    parse_object = context.parse_object
    parse_array = context.parse_array
    parse_string = context.parse_string
    match_number = NUMBER_RE.match
    strict = context.strict
    parse_float = context.parse_float
    parse_int = context.parse_int
    parse_constant = context.parse_constant
    object_hook = context.object_hook
    object_pairs_hook = context.object_pairs_hook
    memo = context.memo

    def _scan_once(string, idx):
        try:
            nextchar = string[idx]
        except IndexError:
            raise StopIteration(idx) from None

        if nextchar == '"':
            return parse_string(string, idx + 1, strict)
        elif nextchar == '{':
            return parse_object((string, idx + 1), strict,
                _scan_once, object_hook, object_pairs_hook, memo)
        elif nextchar == '[':
            return parse_array((string, idx + 1), _scan_once)
        elif nextchar == 'n' and string[idx:idx + 4] == 'null':
            return None, idx + 4
        elif nextchar == 't' and string[idx:idx + 4] == 'true':
            return True, idx + 4
        elif nextchar == 'f' and string[idx:idx + 5] == 'false':
            return False, idx + 5

        m = match_number(string, idx)
        if m is not None:
            integer, frac, exp = m.groups()
            if frac or exp:
                res = parse_float(integer + (frac or '') + (exp or ''))
            else:
                res = parse_int(integer)
            return res, m.end()
        elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
            return parse_constant('NaN'), idx + 3
        elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
            return parse_constant('Infinity'), idx + 8
        elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
            return parse_constant('-Infinity'), idx + 9
        elif nextchar == '+' and string[idx:idx + 9] == '+Infinity':  #  These lines were added;
            return parse_constant('+Infinity'), idx + 9
        else:
            raise StopIteration(idx)

    def scan_once(string, idx):
        try:
            return _scan_once(string, idx)
        finally:
            memo.clear()

    return scan_once

 
class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.scan_once = py_make_scanner(self)  # Force to use python (not c) version of py_make_scnner
 

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
Reply
#6
Thanks scidamn! I had meant to loop back around to that and forgot. There's one small improvement to what you had that I will be incorporating - if line 55 just uses "Infinity" instead of "+Infinity" then you can drop the _CONSTANTS update.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  what type of exception to use? Skaperen 3 997 Sep-30-2022, 05:00 PM
Last Post: Skaperen
  ModuleNotFound but pip shows module installed biscotty666 2 1,511 Jul-14-2022, 05:17 PM
Last Post: Axel_Erfurt
  Os command output in variable shows wrong value paulo79 2 1,467 Apr-09-2022, 03:48 PM
Last Post: ndc85430
Lightbulb trace library xxxlabradorxxx 1 1,118 Oct-01-2021, 11:30 PM
Last Post: Larz60+
  Help in designing a timestamp finder for chapter-less TV shows Daring_T 1 1,819 Oct-26-2020, 09:30 PM
Last Post: Daring_T
  colorbar for scatter shows no negatives values... gil 0 1,514 Apr-15-2020, 12:45 AM
Last Post: gil
  Exception: Returned Type Mismatch Error devansing 1 5,089 Mar-06-2020, 07:26 PM
Last Post: ndc85430
  Type hinting - return type based on parameter micseydel 2 2,425 Jan-14-2020, 01:20 AM
Last Post: micseydel
  How to fix 'uncaught exception of type NSException' in Python MonsterPython 0 2,112 Jul-09-2019, 06:52 AM
Last Post: MonsterPython
  Tracing a multiplication table w/ Python trace() NationalRex22 0 1,728 Jun-11-2019, 03:31 AM
Last Post: NationalRex22

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020