topic Break individual events from json array in Splunk Search

Break individual events from json array

dvarghes — Wed, 30 Sep 2020 04:21:42 GMT

Hello,

I have been working on breaking events which come from the Splunk Rest api addon output. Default "_json" source_type is considering the entire api response as a single event. My aim is to get individual events for objects come under "results". I have tested some custom source types using props.conf but none of them seems working. Please help me with the props.conf entries.

Required individual event example. NB : The fields in the event is dynamic

{
"created" : "2020-02-27T06:14:34Z",
"eventTypeName" : "EVENT1",
"groupId" : "xxx",
"id" : "xxx",
"isGlobalAdmin" : false,
"links" : [ {
"href" : "https://example.com/api/v2",
"rel" : "self"
} ]
}

I am pasting the entire API response below :

Code below :
{
  "links" : [ {
    "href" : "https://example.com/api/v2",
    "rel" : "self"
  } ],
  "results" : [ {
"created" : "2020-02-27T06:14:34Z",
"eventTypeName" : "EVENT1",
"groupId" : "xxxxx",
"id" : "xxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
} ]
}, {
"clusterName" : "splunk-cluster",
"created" : "2020-02-27T06:14:33Z",
"eventTypeName" : "EVENT2",
"groupId" : "xxxxx",
"id" : "xxxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
 } ]
}, {
"created" : "2020-02-27T06:14:32Z",
"eventTypeName" : "EVENT3",
"groupId" : "xxxxxx",
"id" : "xxxxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
} ],
"remoteAddress" : "xx.xx.xx.xx",
"userId" : "xxxxxx",
"username" : "Sam-test"
} ],
"totalCount" : 3
}

Re: Break individual events from json array

dvarghes — Thu, 27 Feb 2020 06:53:20 GMT

The following line breaker is not working :

(\},)

Re: Break individual events from json array

dvarghes — Thu, 27 Feb 2020 10:03:52 GMT

I have tried this in props conf, but not working :

 # props.conf
 [xxxxxxx]
 BREAK_ONLY_BEFORE_DATE = false
 BREAK_ONLY_BEFORE = (\{|\[\s+{)
 MUST_BREAK_AFTER = (\}|\}\s+\])
 SEDCMD-remove_header = s/(\{\s+.+?\[)//g
 SEDCMD-remove_trailing_commas = s/\},/}/g
 SEDCMD-remove_footer = s/\]\s+\}//g

Re: Break individual events from json array

to4kawa — Thu, 27 Feb 2020 21:50:00 GMT

KV_MODE=JSON
only.
OR
https://answers.splunk.com/answers/802709/make-extractions-in-propsconf-from-search-query.html

Re: Break individual events from json array

dvarghes — Fri, 28 Feb 2020 08:42:18 GMT

This did not work. All the API response is being considered as a single event.

Re: Break individual events from json array

to4kawa — Fri, 28 Feb 2020 09:00:32 GMT

spathcan be used single event.
my Q's solution needs LINE_BREAKER

| makeresults 
| eval _raw="{\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}],\"results\":[{\"created\":\"2020-02-27T06:14:34Z\",\"eventTypeName\":\"EVENT1\",\"groupId\":\"xxxxx\",\"id\":\"xxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}]},{\"clusterName\":\"splunk-cluster\",\"created\":\"2020-02-27T06:14:33Z\",\"eventTypeName\":\"EVENT2\",\"groupId\":\"xxxxx\",\"id\":\"xxxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}]},{\"created\":\"2020-02-27T06:14:32Z\",\"eventTypeName\":\"EVENT3\",\"groupId\":\"xxxxxx\",\"id\":\"xxxxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}],\"remoteAddress\":\"xx.xx.xx.xx\",\"userId\":\"xxxxxx\",\"username\":\"Sam-test\"}],\"totalCount\":3}"
| spath

your JSON is valid JSON. KV_MODE=JSON will run.
maybe, your props.conf has extra line breakers, so it's not work.