Python Forum
how do I format json data in splunk?
Thread Rating:
  • 1 Vote(s) - 2 Average
  • 1
  • 2
  • 3
  • 4
  • 5
how do I format json data in splunk?
#1
I wrote a python script in splunk to make API calls and display output to a menu page for my dashboard. I'm not able to get it to display json output. How can I get it to display output formatted correctly. this is the response back from API call --


response = requests.post(url, data=json.dumps(payload))     

print('printing response ', response.json())
Reply
#2
You may try pprint package
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#3
(May-04-2017, 06:55 PM)volcano63 Wrote: I wrote a python script in splunk to make API calls and display output to a menu page for my dashboard
Is this dashboard a website?
Do you want to display the raw JSON,or the more normal approach parse JSON and inserts value into HTML/CSS so it look better?
Reply
#4
(May-04-2017, 09:15 PM)snippsat Wrote:
(May-04-2017, 06:55 PM)volcano63 Wrote: I wrote a python script in splunk to make API calls and display output to a menu page for my dashboard
Is this dashboard a website?
Do you want to display the raw JSON,or the more normal approach parse JSON and inserts value into HTML/CSS so it look better?
Wrong quote!
Quoting @wavic -
Quote:too much vodka
Wink ?!
Test everything in a Python shell (iPython, Azure Notebook, etc.)
  • Someone gave you an advice you liked? Test it - maybe the advice was actually bad.
  • Someone gave you an advice you think is bad? Test it before arguing - maybe it was good.
  • You posted a claim that something you did not test works? Be prepared to eat your hat.
Reply
#5
I'm not able to print python API response in splunk. Can I use ResultsReader to display my response?
Reply
#6
Looks like splunk has an api: http://dev.splunk.com/python
You should use that to shove data into it.  ...unless they read from stdout (is that why you used print?)
Reply
#7
I would like to parse JSON and insert values into HTML/CSS splunk dashboard, so it looks better. i'm using print statements, because my knowledge of python is limited. My dashboard is in HTML/CSS. How can I display what is coming back in my JSON object to a splunk HTML dashboard? I can display output when I run script in my Spyder SDK using simple print commands. But this does not work in splunk, nothing gets printed out on screen.
Reply
#8
I've made some progress, changed the way I make API call., but still having issues with displaying data in splunk. this is part of my code and error message i'm getting. Not sure how to resolve issue?
import requests
import csv
import json
import sys
try: from splunklib.results import *
except: print(sys.exc_info())
try: import httplib
except: print(sys.exc_info())
try: import urllib
except: print(sys.exc_info())

payload = {
     "application": [
       "app_name",
       "description",
       "top_app",
       "status",
       "app_type",
       "gaca_ref",
       "created_on",
       "modified_on",
       "alias_name",
       "business_segment",
       "business_solution",
       "lob",
       "schd_maintenance",
       "schd_outage",
       "vendor_contract",
       "vendor_name",
       "platforms",
       "products",
       "clients",
       "countries"
     ],
     "fields": "custom",
     "uaid_list": ['']}

def getAppMapApplication(uaid_num, user_name, pass_word):
   payload['uaid_list'] = [uaid_num]
   data = loadInputCredentials(user_name, pass_word)
   token = getTokenId(data)
   url2 = "/api/v3/appmap/applications/?format=json&username=abbr3f5&api_token=" + token
   conn = httplib.HTTPSConnection("echo.1dc.com")
   try: conn.request("POST", url2, json.dumps(payload))
   except: print(sys.exc_info())
   try: response = conn.getresponse()
   except: print(sys.exc_info())

   temp = response.read()
   conn.close()
   try: reader = results.ResultsReader(response)
   except: print(sys.exc_info())
   for result in reader:
       print("print result", result.keys())
       if isinstance(result, dict):
           print("Result: %s" % result)
       elif isinstance(result, Message):
           print("Message: %s" % result)
   print("is_preview = %s " % reader.is_preview)


   return response



error message i'm getting:
(<type 'exceptions.NameError'>, NameError("global name 'results' is not defined",), <traceback object at 0x0000000002DB9908>)
Traceback (most recent call last):
 File "C:/Users/tajones/PycharmProjects/appMapAPI/API_calls.py", line 296, in <module>
   main(sys.argv)
 File "C:/Users/tajones/PycharmProjects/appMapAPI/API_calls.py", line 291, in main
   getAppMapApplication(args[2],args[3],args[4])
 File "C:/Users/tajones/PycharmProjects/appMapAPI/API_calls.py", line 195, in getAppMapApplication
   for result in reader:
UnboundLocalError: local variable 'reader' referenced before assignment

Process finished with exit code 1
Reply
#9
I hope this if better>
My API call is working, getting response back. i'm not able to display response using ResultsReader? getting error - exceptions.NameError'>, NameError("global name 'results' is not defined

def getAppMapApplication(uaid_num, user_name, pass_word):
   payload['uaid_list'] = [uaid_num]
   data = loadInputCredentials(user_name, pass_word)
   token = getTokenId(data)
   url2 = "URI string" + token
   conn = httplib.HTTPSConnection("URL server name")
   try: conn.request("POST", url2, json.dumps(payload))
   except: print(sys.exc_info())
   try: response = conn.getresponse()
   except: print(sys.exc_info())

   temp = response.read()
   conn.close()
   try: reader = results.ResultsReader(response)
   except: print(sys.exc_info())
   for result in reader:
       print("print result", result.keys())
       if isinstance(result, dict):
           print("Result: %s" % result)
       elif isinstance(result, Message):
           print("Message: %s" % result)
   print("is_preview = %s " % reader.is_preview)

   return response

error message i'm getting:
(<type 'exceptions.NameError'>, NameError("global name 'results' is not defined",), <traceback object at 0x0000000002DB9908>)
Traceback (most recent call last):
   for result in reader:
UnboundLocalError: local variable 'reader' referenced before assignment

Process finished with exit code 1
Reply
#10
Replace
try: reader = results.ResultsReader(response)
except: print(sys.exc_info())
with
reader = results.ResultsReader(response)
The exception handling doesn't make sense, the program needs to exit if any exception occurs because the variable won't be defined (as the error message says). If you let the exception be thrown, you should get some information that will help with identifying the root issue.
Reply


Possibly Related Threads…
Thread Author Replies Views Last Post
  encrypt data in json file help jacksfrustration 1 220 Mar-28-2024, 05:16 PM
Last Post: deanhystad
  Export data from PDF as tabular format zinho 5 707 Nov-11-2023, 08:23 AM
Last Post: Pedroski55
  format json outputs ! evilcode1 3 1,758 Oct-29-2023, 01:30 PM
Last Post: omemoe277
  How to properly format rows and columns in excel data from parsed .txt blocks jh67 7 1,886 Dec-12-2022, 08:22 PM
Last Post: jh67
  Read nested data from JSON - Getting an error marlonbown 5 1,380 Nov-23-2022, 03:51 PM
Last Post: snippsat
  Convert Json to table format python_student 2 5,550 Sep-28-2022, 12:48 PM
Last Post: python_student
  Reading Data from JSON tpolim008 2 1,093 Sep-27-2022, 06:34 PM
Last Post: Larz60+
  Issue in changing data format (2 bytes) into a 16 bit data. GiggsB 11 2,669 Jul-25-2022, 03:19 PM
Last Post: deanhystad
  Converting cells in excel to JSON format desmondtay 4 1,757 May-23-2022, 10:31 AM
Last Post: Larz60+
  Convert nested sample json api data into csv in python shantanu97 3 2,847 May-21-2022, 01:30 PM
Last Post: deanhystad

Forum Jump:

User Panel Messages

Announcements
Announcement #1 8/1/2020
Announcement #2 8/2/2020
Announcement #3 8/6/2020