topic Re: When building a modular input, how to index JSON data? in Getting Data In

When building a modular input, how to index JSON data?

apezuela — Mon, 09 Apr 2018 11:59:12 GMT

Hi,

I am building a modular input using Add-on Building and python.
When I am trying to index JSON data I get this error: "ERRORcannot serialize {u'rule-number': 1, u'type': u'access-rule..."

Any clue about this issue?

The relevant portion of code is:

res =  json.load(response)   
 rules = res['rulebase']
 for rule in rules:
            event = helper.new_event(source=helper.get_input_type(), index=helper.get_output_index(), sourcetype=helper.get_sourcetype(), data=rule)
        ew.write_event(event)

Best regards,

Re: When building a modular input, how to index JSON data?

p_gurav — Tue, 10 Apr 2018 04:46:40 GMT

This may help:
https://answers.splunk.com/answers/620832/error-execprocessor-message-from-scriptpy-error-ca.html

Re: When building a modular input, how to index JSON data?

smoir_splunk — Wed, 15 Aug 2018 22:43:24 GMT

try json.dump before you write the events

Re: When building a modular input, how to index JSON data?

AndersNierhoff — Mon, 22 Oct 2018 11:09:11 GMT

Hi,

Have you looked at the structure of the raw data ? Splunk modular input is sending data as a xml, and therefor you json parsing fails. i am myself unsure how this is expected to work.

raw event
2018-10-22T13:05:51.329000+0200{'test': 'Issue', 'time': '2018-10-22T13:05:51.329000+0200'}

real event
{'test': 'Issue', 'time': '2018-10-22T13:05:51.329000+0200'}