Hi,
I've been trying to create a new add-on to ingest some data into a new sourcetype within splunk via a REST API service. Unfortunately the api returns JSON as a full array so I am unable to use the add-on creators function to automatically separate arrays into separate events.
I have been doing a lot of reading into event breaking and looked at a number of solutions to this using LINE_BREAKER and BREAK_BEFORE_ONLY however in both instances I haven't succeeded in breaking the event.
The JSON I have been using is [{"idOrg":"abc123","name":"Joe Smith","active":true,"id":"xyz789"},{"idOrg":"efg456","name":"Michael Thomas","active":true,"id":"uvw456"},{"idOrg":"hij789","name":"Craig Lease","active":true,"id":"rst123"}]
I'm trying to get the data into splunk so each nested json is it's own entry. I.e.
{"idOrg":"abc123","name":"Joe Smith","active":true,"id":"xyz789"}
{"idOrg":"abc123","name":"Michael Thomas","active":true,"id":"uvw456"}
{"idOrg":"abc123","name":"Craig Lease","active":true,"id":"rst123"}
I have managed to get this to work as intended when importing this data via the sourcetype editor using the following in props.conf
However when running this outside the add-on creator on my machine, I cannot get the same level of success.
It's worthwhile noting that I have the following commands in my props.conf in addition to the ones above:
So far I have taken the following actions in order to attempt to resolve this. All of which have had no benefit.
\}(,)\{
},
with }NEWLINE
and change LINE_BREAKER to NEWLINE
. The SEDCMD worked as expected but no luck with LINE_BREAKERAny advice at this stage would be very gratefully received.
Regards
Tom
Hi,
Thankyou very much for your suggestion, however unfortunately it made no difference.
The answer in the end was as simple as editing the data input within the add-on creator and specify the JSON path as$.[*]
within the event extraction settings.
Thanks
Tom
Hi,
Thankyou very much for your suggestion, however unfortunately it made no difference.
The answer in the end was as simple as editing the data input within the add-on creator and specify the JSON path as$.[*]
within the event extraction settings.
Thanks
Tom
Hi @tomawest,
I am facing similar situation where I am trying to extract events from newrelic addon and unable to perform line break for JSON lines from single event to multiple events. Could you please help me explain with stanza configuration and how it is done like which conf file and what stanza i need to add JSON path $.[*]
?
Thanks,
Shashank Soni
Hi there,
Did you try below combination,
BREAK_ONLY_BEFORE = (?m)\{\"idOrg\"\:
SHOULD_LINEMERGE = true