How to parse key-value pairs from logs in JSON format?

[rameshlpatel]

Hi, In my Java application, I am printing logs in JSON format. Here in JSON "message" field I am logging a value as key-value pair format (shown below).

message: {"TOKEN_ID":"5122b288-9201-4826-93e0-325ad7f2aa00","Result":"SUCCESS","Method":"validateToken()","DATE":"Thu Feb 26 04:42:30 CST 2015","REALM":"LDAP_AD","EMPLOYEE_ID":"999999"}

However SPLUNK parser is not able to parse them as SPLUNK fields like Result=SUCCESS, EMPLOYEE_ID=99999 etc. Please help me in this issue.

{ [-] 
   context: default 
   level: INFO 
   logger: com.wm.ocs.driverSecurity.service.impl.TokenValidationServiceImpl 
   message: {"TOKEN_ID":"5122b288-9201-4826-93e0-325ad7f2aa00","Result":"SUCCESS","Method":"validateToken()","DATE":"Thu Feb 26 04:42:30 CST 2015","REALM":"LDAP_AD","EMPLOYEE_ID":"999999"} 
   thread: WebContainer : 3 
   timestamp: 2015-02-26T04:42:30.456Z 
}

[cpetterborg]

You can get all the values from the JSON string by setting the props.conf to know that the data is JSON formatted. If it is not completely JSON formatted, however, it will not work. In otherwords, the JSON string must be the only thing in the event. Even the date string must be found within the JSON string. When that works, you get the same effect as with KV pairs.

I've had success with setting my props.conf for the JSON data to have the following:

KV_MODE = json
SHOULD_LINEMERGE = true

The SHOULD_LINEMERGE should be set according to your events, where true would be for multi-line events.

Accessing the fields is simple enough where you go through the name by the hierarchy of the string. In your event, you might use something like message.EMPLOYEE_ID to access that field from the JSON string. See the spath documentation (http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Spath) for more information.

[somesoni2]

By default, Splunk doesn't treat ":" as KV delimiter. Have a look at following post to see what you need to configure to achieve that.

http://answers.splunk.com/answers/34943/key-value-pair-extraction.html

[rameshlpatel]

I have also tried as "=" as KV delimiter but SPLUNK considering as string value for message field. I want SPLUNK to consider that as inner JSON string and make it as KV pair.

Is there any way to configure splunk to parse inner JSON string ?

[dominiquevocat]

something like this? | extract pairdelim=",", kvdelim=":"
it will treat the field _raw only for all i know