How to break events from JSON data.

[shanksholla]

Hi

I have JSON data which I have to break into events. The data looks somewhat like:

{    "data": [       {          "id": "X999_Y999",          "from": {             "name": "foo Brady", "id": "P12"         },          "message": "Looking forward to 2010!",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X999/posts/Y999"            },             {                "name": "Like",                "link": "http://www.facebook.com/M999/posts/Z999"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      },       {          "id": "A998_Z998",          "from": {             "name": "foo Manning", "id": "P18"         },          "message": "Where's my contract?",          "actions": [             {                "name": "Comment",                "link": "http://www.facebook.com/X998/posts/Z998"            },             {                "name": "Like",                "link": "http://www.facebook.com/X998/posts/Z998"            }          ],          "type": "status",          "created_time": "2010-08-02T21:27:44+0000",          "updated_time": "2010-08-02T21:27:44+0000"      }    ] }

I need to break this message into events at "id": "A998_Z998". That is-
}, { "id":
from this JSON message. I have tried the following for the props.conf-

BREAK_ONLY_BEFORE=,\s*{\s*"id":
INDEXED_EXTRACTIONS=json
KV_MODE=none
SHOULD_LINEMERGE=true
disabled=false
pulldown_type=true

The JSON key value pairs are extracted, but I'm not able to break the JSON message into events at the required place. Can you please help me with this?

[tskinnerivsec]

Did you try escaping some of the literal characters in your regex: BREAK_ONLY_BEFORE=,s*{s*"id":

this should be:

BREAK_ONLY_BEFORE = \,\s*\{\s*\"id\"\:

actually, I would also use \s+ instead of \s*, if you know that there will be at least one space character. The BREAK_ONLY_BEFORE directive
specifies a regex pattern, what you had in your configuration example wasn't regex, it was just a string of literal characters. \s is a space character in regex, not s. This answer is assuming what is in your example is exactly what's in your props.conf config and splunk answers didn't strip out the escape characters out for you when you posted it.

[tskinnerivsec]

did you try setting KV_MODE=json?

[tskinnerivsec]

instead of using one or more space characters in your regex after the { character, try using a negative character class instead like so:

\,\s+\{[^\"]+\"id\"\:

[shanksholla]

Thanks for your reply. But this didn't work. Though all these regexes work fine in any regex tool, Splunk is not recognizing them. Basically, now all the data comes in one event.

[shanksholla]

Sorry! There was a typo in props.conf properties I gave. I meant-

BREAK_ONLY_BEFORE=\,\s*\{\s*"id":

I guess splunk answers stripped the escape characters. Basically I have used regex for white space.Hope BREAK_ONLY_BEFORE statement turned out right this time 🙂
Also, as suggested have used s+ instead of s*. But no luck with the event split.