How can I split the HTTP JSON response into multiple events?
My REST API is returning a JSON Array and for each array element I like to create separate event in Splunk.
I tried sys.stdout.flush() but without success. How can I set event boundaries?
This is my json response
[{"sever1": true}, {"sever2": true}]
This is my run method
def run_script():
try:
cfg = get_config()
conn = httplib.HTTPConnection(cfg["url"])
conn.request("GET", "/")
r1 = conn.getresponse()
data1 = json.loads(r1.read())
for d in data1:
print d
sys.stdout.flush()
conn.close()
except Exception as e:
logger.error(e)
logger.info("RUN HTTP DONE")
Each array element should be a separate event.
I solved this problem. I overlooked the SCHEMA config.
It's important to set the streaming mode to xml
SCHEME = """<scheme>
<title>Hello</title>
<description>Log Data to Splunk</description>
<streaming_mode>xml</streaming_mode>
</scheme>
"""
# Empty introspection routine
def do_scheme():
print SCHEME
See Documentation:
https://docs.splunk.com/Documentation/Splunk/6.6.2/AdvancedDev/ModInputsStream
And than I can post separate events to Splunk by using this code.
# prints XML stream
def post_data(time, data):
print "<stream><event unbroken=\"1\"><time>%s</time><data>%s</data><done/></event></stream>" % (time,data)
It's also important to set unbroken attribute and and the tag.
See Documentation:
https://docs.splunk.com/Documentation/Splunk/6.6.2/AdvancedDev/ModInputsStream
I've moved your comment to an answer. Please accept your own answer here so that it will be flagged as having an accepted answer. - Thanks!
Is your JSON data actually all on one line that you want to separate into different events? That is, is it like this?:
[{"sever1": true}, {"sever2": true}]
or this?:
[{"sever1": true},
{"sever2": true}]
What is your purpose in separating the values into different events? This sort of thing can be done at search time without having to do it at index time.