Splunk Search

Break individual events from json array

dvarghes
Explorer

Hello,

I have been working on breaking events which come from the Splunk Rest api addon output. Default "_json" source_type is considering the entire api response as a single event. My aim is to get individual events for objects come under "results". I have tested some custom source types using props.conf but none of them seems working. Please help me with the props.conf entries.

Required individual event example. NB : The fields in the event is dynamic

{
"created" : "2020-02-27T06:14:34Z",
"eventTypeName" : "EVENT1",
"groupId" : "xxx",
"id" : "xxx",
"isGlobalAdmin" : false,
"links" : [ {
"href" : "https://example.com/api/v2",
"rel" : "self"
} ]
}

I am pasting the entire API response below :

Code below :
{
  "links" : [ {
    "href" : "https://example.com/api/v2",
    "rel" : "self"
  } ],
  "results" : [ {
"created" : "2020-02-27T06:14:34Z",
"eventTypeName" : "EVENT1",
"groupId" : "xxxxx",
"id" : "xxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
} ]
}, {
"clusterName" : "splunk-cluster",
"created" : "2020-02-27T06:14:33Z",
"eventTypeName" : "EVENT2",
"groupId" : "xxxxx",
"id" : "xxxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
 } ]
}, {
"created" : "2020-02-27T06:14:32Z",
"eventTypeName" : "EVENT3",
"groupId" : "xxxxxx",
"id" : "xxxxxx",
"isGlobalAdmin" : false,
"links" : [ {
  "href" : "https://example.com/api/v2",
  "rel" : "self"
} ],
"remoteAddress" : "xx.xx.xx.xx",
"userId" : "xxxxxx",
"username" : "Sam-test"
} ],
"totalCount" : 3
}
0 Karma

dvarghes
Explorer

The following line breaker is not working :

(\},)
0 Karma

dvarghes
Explorer

I have tried this in props conf, but not working :

 # props.conf
 [xxxxxxx]
 BREAK_ONLY_BEFORE_DATE = false
 BREAK_ONLY_BEFORE = (\{|\[\s+{)
 MUST_BREAK_AFTER = (\}|\}\s+\])
 SEDCMD-remove_header = s/(\{\s+.+?\[)//g
 SEDCMD-remove_trailing_commas = s/\},/}/g
 SEDCMD-remove_footer = s/\]\s+\}//g
0 Karma

to4kawa
Ultra Champion
0 Karma

dvarghes
Explorer

This did not work. All the API response is being considered as a single event.

0 Karma

to4kawa
Ultra Champion

spathcan be used single event.
my Q's solution needs LINE_BREAKER

| makeresults 
| eval _raw="{\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}],\"results\":[{\"created\":\"2020-02-27T06:14:34Z\",\"eventTypeName\":\"EVENT1\",\"groupId\":\"xxxxx\",\"id\":\"xxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}]},{\"clusterName\":\"splunk-cluster\",\"created\":\"2020-02-27T06:14:33Z\",\"eventTypeName\":\"EVENT2\",\"groupId\":\"xxxxx\",\"id\":\"xxxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}]},{\"created\":\"2020-02-27T06:14:32Z\",\"eventTypeName\":\"EVENT3\",\"groupId\":\"xxxxxx\",\"id\":\"xxxxxx\",\"isGlobalAdmin\":false,\"links\":[{\"href\":\"https://example.com/api/v2\",\"rel\":\"self\"}],\"remoteAddress\":\"xx.xx.xx.xx\",\"userId\":\"xxxxxx\",\"username\":\"Sam-test\"}],\"totalCount\":3}"
| spath

your JSON is valid JSON. KV_MODE=JSON will run.
maybe, your props.conf has extra line breakers, so it's not work.

Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...