Getting Data In

How to parse key-value pairs from logs in JSON format?

rameshlpatel
Communicator

Hi, In my Java application, I am printing logs in JSON format. Here in JSON "message" field I am logging a value as key-value pair format (shown below).

message: {"TOKEN_ID":"5122b288-9201-4826-93e0-325ad7f2aa00","Result":"SUCCESS","Method":"validateToken()","DATE":"Thu Feb 26 04:42:30 CST 2015","REALM":"LDAP_AD","EMPLOYEE_ID":"999999"}

However SPLUNK parser is not able to parse them as SPLUNK fields like Result=SUCCESS, EMPLOYEE_ID=99999 etc. Please help me in this issue.

{ [-] 
   context: default 
   level: INFO 
   logger: com.wm.ocs.driverSecurity.service.impl.TokenValidationServiceImpl 
   message: {"TOKEN_ID":"5122b288-9201-4826-93e0-325ad7f2aa00","Result":"SUCCESS","Method":"validateToken()","DATE":"Thu Feb 26 04:42:30 CST 2015","REALM":"LDAP_AD","EMPLOYEE_ID":"999999"} 
   thread: WebContainer : 3 
   timestamp: 2015-02-26T04:42:30.456Z 
}
Tags (3)
0 Karma

cpetterborg
SplunkTrust
SplunkTrust

You can get all the values from the JSON string by setting the props.conf to know that the data is JSON formatted. If it is not completely JSON formatted, however, it will not work. In otherwords, the JSON string must be the only thing in the event. Even the date string must be found within the JSON string. When that works, you get the same effect as with KV pairs.

I've had success with setting my props.conf for the JSON data to have the following:

KV_MODE = json
SHOULD_LINEMERGE = true

The SHOULD_LINEMERGE should be set according to your events, where true would be for multi-line events.

Accessing the fields is simple enough where you go through the name by the hierarchy of the string. In your event, you might use something like message.EMPLOYEE_ID to access that field from the JSON string. See the spath documentation (http://docs.splunk.com/Documentation/Splunk/6.2.1/SearchReference/Spath) for more information.

somesoni2
Revered Legend

By default, Splunk doesn't treat ":" as KV delimiter. Have a look at following post to see what you need to configure to achieve that.

http://answers.splunk.com/answers/34943/key-value-pair-extraction.html

0 Karma

rameshlpatel
Communicator

I have also tried as "=" as KV delimiter but SPLUNK considering as string value for message field. I want SPLUNK to consider that as inner JSON string and make it as KV pair.

Is there any way to configure splunk to parse inner JSON string ?

0 Karma

dominiquevocat
SplunkTrust
SplunkTrust

something like this? | extract pairdelim=",", kvdelim=":"
it will treat the field _raw only for all i know

0 Karma
Get Updates on the Splunk Community!

Splunk Observability Cloud’s AI Assistant in Action Series: Analyzing and ...

This is the second post in our Splunk Observability Cloud’s AI Assistant in Action series, in which we look at ...

Elevate Your Organization with Splunk’s Next Platform Evolution

 Thursday, July 10, 2025  |  11AM PDT / 2PM EDT Whether you're managing complex deployments or looking to ...

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...