Python Forum
json.load causing IndexError: list index out of range
Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
json.load causing IndexError: list index out of range
#1
I'm trying to debug the ubuntu-mate-welcome python script which is refusing to load a valid json file.

I have validated the json file with jsonlint. However my experience with python is limited.

My environment is Ubuntu 18.04 aarch64 (arm64).

I have isolated the problem code in an example included here:

#! /usr/bin/python3
import sys
import json

print(sys.path)

json_path = ''

try:
    print('Reading index...')
    json_path = '/usr/share/ubuntu-mate-welcome/js/applications.json'
    with open(json_path) as data_file:
        print('Loading JSON.')
        index = json.load(data_file)
        print('LOADED JSON!!!!')
except Exception as e:
    index = None
    print('Software Index JSON is invalid or missing!')
    print(json_path)
The output is as follows:

Output:
['/home/odroid/Scripts', '/usr/lib/python36.zip', '/usr/lib/python3.6', '/usr/lib/python3.6/lib-dynload', '/usr/local/lib/python3.6/dist-packages', '/usr/lib/python3/dist-packages'] Reading index... Loading JSON. Software Index JSON is invalid or missing! /usr/share/ubuntu-mate-welcome/js/applications.json
However my example does not display the :

Error:
IndexError: list index out of range
So I am not certain I have faithfully reproduced the problem!

Output from jsonlint-php:

Output:
jsonlint-php /usr/share/ubuntu-mate-welcome/js/applications.json Valid JSON (/usr/share/ubuntu-mate-welcome/js/applications.json)
I have looked in the python path but cannot identify the json module referred to in the import statement.

How can I be certain I have the json module installed?
Reply
#2
First step in this case is to remove the try/except. At the moment you catch all exceptions.
Then post full traceback you get as well as the json file itself.

(May-19-2019, 01:45 PM)WallyZ Wrote: How can I be certain I have the json module installed?
it's part of the python standard library, so you have it
If you can't explain it to a six year old, you don't understand it yourself, Albert Einstein
How to Ask Questions The Smart Way: link and another link
Create MCV example
Debug small programs

Reply
#3
raise the exception again to see where the error occurs.
You could try https://github.com/codecobblers/dirtyjson

I downloaded the package and extracted the file: https://packages.ubuntu.com/xenial/all/u...te-welcome
I am able to load the json with Python 3.7.3

You can also upload your broken applications.json.
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#4
Ok after removing the try/except I now get a different picture:

Error:
Traceback (most recent call last): File "/home/odroid/Scripts/python_import_paths.py", line 14, in <module> index = json.load(data_file) File "/usr/lib/python3.6/json/__init__.py", line 296, in load return loads(fp.read(), File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode return codecs.ascii_decode(input, self.errors)[0] UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 210343: ordinal not in range(128)
Does that mean there is a strange character in the json file?

Found some accented characters in the json file. The application Deja had accented e and a.

I changed them to normal a nad e and the json now loads. This is the standard applications.json for the ubuntu-mate-welcome package which loads OK on my Ubuntu 18.04 Mate Raspberry PI system.

So why does my code fail on one system and succeed on the other when both applications.json files are the same?

So is there some python setting I can set to make accented characters acceptable?

Why are accented characters acceptable on my Ubuntu 18.04 PI but unacceptable on my Ubuntu 18.04 Odroid?
Reply
#5
The file seems to be corrupt.

Usually json saves unicode chars with the prefix \u followed by a hex number.
If the character is encoded with utf8 (no \u notation), the normal json module is still able to load and decode it.
But if there for example quotes are missing, this can lead into this strange exception.

You could try the dirtyjson, load the string and dump it with the python-json module. Then
you can compare the original file together with the corrected fie (from json.dump or json.dumps).
Almost dead, but too lazy to die: https://sourceserver.info
All humans together. We don't need politicians!
Reply
#6
Thanks for all the advise.

I did test the json file in both jsonlint and dirty json. Both tools stated the json was valid.

Here is snippet of the json file where the accented characters are found:

"deja-dup": {
"name": "Déjà Dup",
"img": "deja-dup",
"main-package": "deja-dup",
"launch-command": "deja-dup",

I just decide to replace the accented character in the end.

I thought it may have been a python 2 versus 3 issue but I tried both versions. The script doesn't appear to work at all in version 2 so I was suprised that the accented characters caused an issue in version 3.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  JSON Dump and JSON Load foxholenoob 8 977 Oct-12-2023, 07:21 AM
Last Post: foxholenoob
  IndexError: index 10 is out of bounds for axis 0 with size 10 Mehboob 11 1,952 Sep-14-2023, 06:54 AM
Last Post: Mehboob
Thumbs Down I hate "List index out of range" Melen 20 3,161 May-14-2023, 06:43 AM
Last Post: deanhystad
  Response.json list indices must be integers or slices, not str [SOLVED] AlphaInc 4 6,185 Mar-24-2023, 08:34 AM
Last Post: fullytotal
Exclamation IndexError: Replacement index 2 out of range for positional args tuple - help? MrKnd94 2 5,967 Oct-14-2022, 09:57 PM
Last Post: MrKnd94
  IndexError: list index out of range dolac 4 1,844 Jul-25-2022, 03:42 PM
Last Post: deanhystad
  I'm getting a String index out of range error debian77 7 2,280 Jun-26-2022, 09:50 AM
Last Post: deanhystad
  IndexError: list index out of range Anldra12 2 1,409 May-03-2022, 01:39 PM
Last Post: Anldra12
  TypeError: list indices must be integers or slices, not range Anldra12 2 2,501 Apr-22-2022, 10:56 AM
Last Post: Anldra12
  matplotlib x axis range goes over the set range Pedroski55 5 3,110 Nov-21-2021, 08:40 AM
Last Post: paul18fr

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020