Splunk Search

How to break events from JSON data.

shanksholla
Explorer

Hi

I have JSON data which I have to break into events. The data looks somewhat like:

{    "data": [       {          "id": "X999_Y999",          "from": {             "name": "foo Brady", "id": "P12"         },          "message": "Looking forward to 2010!",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X999/posts/Y999"            },             {                "name": "Like",                "link": "http://www.facebook.com/M999/posts/Z999"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      },       {          "id": "A998_Z998",          "from": {             "name": "foo Manning", "id": "P18"         },          "message": "Where's my contract?",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X998/posts/Z998"            },             {                "name": "Like",                "link": "http://www.facebook.com/X998/posts/Z998"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      }    ] }

I need to break this message into events at "id": "A998_Z998". That is-
}, { "id":
from this JSON message. I have tried the following for the props.conf-

BREAK_ONLY_BEFORE=,\s*{\s*"id":
INDEXED_EXTRACTIONS=json
KV_MODE=none
SHOULD_LINEMERGE=true
disabled=false
pulldown_type=true

The JSON key value pairs are extracted, but I'm not able to break the JSON message into events at the required place. Can you please help me with this?

0 Karma

tskinnerivsec
Contributor

Did you try escaping some of the literal characters in your regex: BREAK_ONLY_BEFORE=,s*{s*"id":

this should be:

BREAK_ONLY_BEFORE = \,\s*\{\s*\"id\"\:

actually, I would also use \s+ instead of \s*, if you know that there will be at least one space character. The BREAK_ONLY_BEFORE directive
specifies a regex pattern, what you had in your configuration example wasn't regex, it was just a string of literal characters. \s is a space character in regex, not s. This answer is assuming what is in your example is exactly what's in your props.conf config and splunk answers didn't strip out the escape characters out for you when you posted it.

0 Karma

tskinnerivsec
Contributor

did you try setting KV_MODE=json?

0 Karma

tskinnerivsec
Contributor

instead of using one or more space characters in your regex after the { character, try using a negative character class instead like so:

\,\s+\{[^\"]+\"id\"\:
0 Karma

shanksholla
Explorer

Thanks for your reply. But this didn't work. Though all these regexes work fine in any regex tool, Splunk is not recognizing them. Basically, now all the data comes in one event.

0 Karma

shanksholla
Explorer

Sorry! There was a typo in props.conf properties I gave. I meant-

BREAK_ONLY_BEFORE=\,\s*\{\s*"id":

I guess splunk answers stripped the escape characters. Basically I have used regex for white space.Hope BREAK_ONLY_BEFORE statement turned out right this time 🙂
Also, as suggested have used s+ instead of s*. But no luck with the event split.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...