Bottom Page

Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
 Stack trace shows different exception type than print
#1
Python's JSON module supports deserializing Javascript infinities (although the official JSON spec doesn't). Specifically, it supports "Infinity" and "-Infinity" in particular. I need to be able to deserialize "+Infinity" though. Here is my attempt at achieving this
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except json.decoder.JSONDecodeError as e:
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            raise e
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
The result of this is that (apparently) a StopIteration exception is thrown, although its traceback looks like a JSONDecodeError
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... Got an unexpected exception from default (type: <class 'StopIteration'>) - 0 Traceback (most recent call last): File "confusion.py", line 28, in <module> print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder)) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/__init__.py", line 361, in loads return cls(**kw).decode(s) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/json/decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
I'm baffled because the stack trace clearly shows a JSONDecodeError being raised, but at runtime when I try to catch it, it appears to have a different type. In case the class is doing something funky, I also tried catching a ValueError (its superclass) but that didn't work either.

What does work is catching a StopIteration exception, but that doesn't seem right. I'm afraid that there's something weird going on here and if I catch the wrong exception, there will be some surprise down the line. Does anyone know what's going on here?
Feel like you're not getting the answers you want? Checkout the help/rules for things like what to include/not include in a post, how to use code tags, how to ask smart questions, and more.

Pro-tip - there's an inverse correlation between the number of lines of code posted and my enthusiasm for helping with a question :)
Quote
#2
Henlo Fren,

My naem Sheba and I luvs da slithery codes please.

The JSONDecodeError in your codes is from JSONDecoder.raw_decode which calls self.scan_once.

Here is a good codes please:
import json

class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.original_scan_once = self.scan_once
        self.scan_once = self._scan_once

    def _scan_once(self, string, idx):
        print("Custom _scan_once entered; trying default first...")
        try:
            return self.original_scan_once(string, idx)
        except StopIteration as e:  # scan_once should throw a StopIteration
            print("Default failed, trying custom logic...")
            try:
                nextchar = string[idx]
            except IndexError:
                raise StopIteration(idx) from e
            if nextchar == '+' and string[idx:idx + 9] == '+Infinity':
                return self.parse_constant('Infinity'), idx + 9
            else:
                raise StopIteration(idx)
        except Exception as e:
            print(f"Got an unexpected exception from default (type: {type(e)}) - {e}")
            raise e

print(json.loads('Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))
print()
print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder))
print()
Output:
$ python3 confusion.py Custom _scan_once entered; trying default first... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... inf Custom _scan_once entered; trying default first... Default failed, trying custom logic... Traceback (most recent call last): File "confusion.py", line 31, in <module> print(json.loads('+BadData', cls=PositiveInfinityJSONDecoder)) File "/usr/lib/python3.6/json/__init__.py", line 367, in loads return cls(**kw).decode(s) File "/usr/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/usr/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
Sincerely,
Sheba Weeba Possum Woo
Quote
#3
Thanks for the reply but I feel like I must be missing something. I already mentioned that a StopIteration error appears to work, but that's the problem. As you can see in the link you provided, raw_decode should be throwing a JSONDecodeError, not a StopIteration error, and you can see that when I print a stacktrace the type of the error appears different than if I print the type of the error object.
Feel like you're not getting the answers you want? Checkout the help/rules for things like what to include/not include in a post, how to use code tags, how to ask smart questions, and more.

Pro-tip - there's an inverse correlation between the number of lines of code posted and my enthusiasm for helping with a question :)
Quote
#4
There are two exceptions being thrown:
  1. json.loads('+Infinity') is called.
  2. ...
  3. raw_decode('+Infinity') is called.
  4. raw_decode calls PositiveInfinityJSONDecoder._scan_once.
  5. PositiveInfinityJSONDecoder._scan_once calls self.original_scan_once (a.k.a. JSONDecoder().scan_once).
  6. self.original_scan_once raises a StopIteration as expected.
  7. PositiveInfinityJSONDecoder._scan_once prints the StopIteration exception and reraises it.
  8. raw_decode excepts StopIteration and raises JSONDecodeError.
  9. JSONDecodeError is not caught, so it prints the stack trace for JSONDecodeError.
Quote
#5
There are two version of make_scanner in json module, c_make_scanner and py_make_scanner. We need to force Python to use Python version (py_make_scanner).
The following isn't to be an elegant solution, but it straightforward and clear:

import json
from json.decoder import PosInf, _CONSTANTS
from json.scanner import NUMBER_RE

_CONSTANTS.update({'+Infinity': PosInf})  # Update known constants 

def py_make_scanner(context):  # there are c-version, we need to force python to use version only
    parse_object = context.parse_object
    parse_array = context.parse_array
    parse_string = context.parse_string
    match_number = NUMBER_RE.match
    strict = context.strict
    parse_float = context.parse_float
    parse_int = context.parse_int
    parse_constant = context.parse_constant
    object_hook = context.object_hook
    object_pairs_hook = context.object_pairs_hook
    memo = context.memo

    def _scan_once(string, idx):
        try:
            nextchar = string[idx]
        except IndexError:
            raise StopIteration(idx) from None

        if nextchar == '"':
            return parse_string(string, idx + 1, strict)
        elif nextchar == '{':
            return parse_object((string, idx + 1), strict,
                _scan_once, object_hook, object_pairs_hook, memo)
        elif nextchar == '[':
            return parse_array((string, idx + 1), _scan_once)
        elif nextchar == 'n' and string[idx:idx + 4] == 'null':
            return None, idx + 4
        elif nextchar == 't' and string[idx:idx + 4] == 'true':
            return True, idx + 4
        elif nextchar == 'f' and string[idx:idx + 5] == 'false':
            return False, idx + 5

        m = match_number(string, idx)
        if m is not None:
            integer, frac, exp = m.groups()
            if frac or exp:
                res = parse_float(integer + (frac or '') + (exp or ''))
            else:
                res = parse_int(integer)
            return res, m.end()
        elif nextchar == 'N' and string[idx:idx + 3] == 'NaN':
            return parse_constant('NaN'), idx + 3
        elif nextchar == 'I' and string[idx:idx + 8] == 'Infinity':
            return parse_constant('Infinity'), idx + 8
        elif nextchar == '-' and string[idx:idx + 9] == '-Infinity':
            return parse_constant('-Infinity'), idx + 9
        elif nextchar == '+' and string[idx:idx + 9] == '+Infinity':  #  These lines were added;
            return parse_constant('+Infinity'), idx + 9
        else:
            raise StopIteration(idx)

    def scan_once(string, idx):
        try:
            return _scan_once(string, idx)
        finally:
            memo.clear()

    return scan_once

 
class PositiveInfinityJSONDecoder(json.JSONDecoder):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        self.scan_once = py_make_scanner(self)  # Force to use python (not c) version of py_make_scnner
 

print(json.loads('+Infinity', cls=PositiveInfinityJSONDecoder))


micseydel likes this post
Quote
#6
Thanks scidamn! I had meant to loop back around to that and forgot. There's one small improvement to what you had that I will be incorporating - if line 55 just uses "Infinity" instead of "+Infinity" then you can drop the _CONSTANTS update.
Feel like you're not getting the answers you want? Checkout the help/rules for things like what to include/not include in a post, how to use code tags, how to ask smart questions, and more.

Pro-tip - there's an inverse correlation between the number of lines of code posted and my enthusiasm for helping with a question :)
Quote

Top Page

Possibly Related Threads...
Thread Author Replies Views Last Post
  Type hinting - return type based on parameter micseydel 2 122 Jan-14-2020, 01:20 AM
Last Post: micseydel
  How to fix 'uncaught exception of type NSException' in Python MonsterPython 0 296 Jul-09-2019, 06:52 AM
Last Post: MonsterPython
  Tracing a multiplication table w/ Python trace() NationalRex22 0 245 Jun-11-2019, 03:31 AM
Last Post: NationalRex22
  after using openpyxl to add colors to script, black shows up white online in excel Soundtechscott 1 577 Jun-08-2019, 10:33 PM
Last Post: Soundtechscott
  trace invalid pointer simon149 7 1,062 Apr-16-2019, 07:05 AM
Last Post: simon149
  How to create shadow between two colours and control a line after we trace it? CrazyPythonNerd 0 301 Mar-06-2019, 01:31 PM
Last Post: CrazyPythonNerd
  During handling of the above exception, another exception occurred Skaperen 7 9,098 Dec-21-2018, 10:58 AM
Last Post: Gribouillis
  choice of exception type Skaperen 1 696 Sep-01-2018, 05:50 AM
Last Post: Gribouillis
  selecting a particular column in csv file shows error raady07 7 1,174 Mar-15-2018, 02:19 PM
Last Post: Larz60+
  Debug and trace in python quocchi22101262 0 1,558 Jun-20-2017, 09:35 AM
Last Post: quocchi22101262

Forum Jump:


Users browsing this thread: 1 Guest(s)