Splunk Search

How to break events from JSON data.

Explorer

Hi

I have JSON data which I have to break into events. The data looks somewhat like:

{    "data": [       {          "id": "X999_Y999",          "from": {             "name": "foo Brady", "id": "P12"         },          "message": "Looking forward to 2010!",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X999/posts/Y999"            },             {                "name": "Like",                "link": "http://www.facebook.com/M999/posts/Z999"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      },       {          "id": "A998_Z998",          "from": {             "name": "foo Manning", "id": "P18"         },          "message": "Where's my contract?",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X998/posts/Z998"            },             {                "name": "Like",                "link": "http://www.facebook.com/X998/posts/Z998"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      }    ] }

I need to break this message into events at "id": "A998_Z998". That is-
}, { "id":
from this JSON message. I have tried the following for the props.conf-

BREAK_ONLY_BEFORE=,\s*{\s*"id":
INDEXED_EXTRACTIONS=json
KV_MODE=none
SHOULD_LINEMERGE=true
disabled=false
pulldown_type=true

The JSON key value pairs are extracted, but I'm not able to break the JSON message into events at the required place. Can you please help me with this?

0 Karma

Contributor

Did you try escaping some of the literal characters in your regex: BREAKONLYBEFORE=,s{s"id":

this should be:

BREAK_ONLY_BEFORE = \,\s*\{\s*\"id\"\:

actually, I would also use \s+ instead of \s*, if you know that there will be at least one space character. The BREAKONLYBEFORE directive
specifies a regex pattern, what you had in your configuration example wasn't regex, it was just a string of literal characters. \s is a space character in regex, not s. This answer is assuming what is in your example is exactly what's in your props.conf config and splunk answers didn't strip out the escape characters out for you when you posted it.

0 Karma

Contributor

did you try setting KV_MODE=json?

0 Karma

Contributor

instead of using one or more space characters in your regex after the { character, try using a negative character class instead like so:

\,\s+\{[^\"]+\"id\"\:
0 Karma

Explorer

Thanks for your reply. But this didn't work. Though all these regexes work fine in any regex tool, Splunk is not recognizing them. Basically, now all the data comes in one event.

0 Karma

Explorer

Sorry! There was a typo in props.conf properties I gave. I meant-

BREAK_ONLY_BEFORE=\,\s*\{\s*"id":

I guess splunk answers stripped the escape characters. Basically I have used regex for white space.Hope BREAKONLYBEFORE statement turned out right this time 🙂
Also, as suggested have used s+ instead of s*. But no luck with the event split.

0 Karma