Splunk Search

Help with extracting JSON fields

kranthimutyala
Path Finder

Hi Team,
I have the event in the below format and want to extract the key-value pairs as fields.

Please help extract fields from LogDate till the user.Thanks

 

 

{ [-]
   event: INFO  2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-ust-email-notification-v1-uw-qa].get:\ping:Router.CPU_LITE @6c1fb7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {
  "LogDate": "09/23/2022 16:11:13.932",
  "LogNo": "99",
  "LogLevel": "INFO",
  "LogType": "Process Level",
  "LogMessage": "Splunk anypoint log",
  "TimeTaken": "0:00:12.628",
  "ProcessName": "AnypointSplunkTest",
  "TaskName": "AnypointTest",
  "RPAEnvironment": "DEV",
  "LogId": "002308900.20250824210419999",
  "MachineName": "abc-xyz-efg",
  "User": "name.first"
}
   metaData: { [+]
   }
}

 

 

 

and this is the raw text 

{"metaData":{"sourceApiVersion":"1.0.0-SNAPSHOT","index":"aas","sourceApi":"papi-cust-email-notification-v1-uw-qa","cloudhubEnvironment":"AUTOMATION-QA","tags":""},"event":"INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {\n \"LogDate\": \"09/23/2022 16:11:13.932\",\n \"LogNo\": \"99\",\n \"LogLevel\": \"INFO\",\n \"LogType\": \"Process Level\",\n \"LogMessage\": \"Splunk anypoint log\",\n \"TimeTaken\": \"0:00:12.628\",\n \"ProcessName\": \"AnypointSplunkTest\",\n \"TaskName\": \"AnypointTest\",\n \"RPAEnvironment\": \"DEV\",\n \"LogId\": \"002308900.20250824210419999\",\n \"MachineName\": \"abc-xyz-wd\",\n \"User\": \"name.first\"\n}"}

Labels (3)
Tags (3)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

 

| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG

 

Your sample data now gives

LOGLogDateLogIdLogLevelLogMessageLogNoLogTypeMachineNameProcessNameRPAEnvironmentTaskNameTimeTaken 
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }09/23/2022 16:11:13.932002308900.20250824210419999INFOSplunk anypoint log99Process Levelabc-xyz-wdAnypointSplunkTestDEVAnypointTest0If:00:12.628name.first
If Splunk doesn't give you event field, apply spath first to extract event.

 

| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG​

 

 

View solution in original post

Tags (1)
0 Karma

johnhuang
Motivator

This should work:

| rex "(?<_raw>\"LogDate[^\}]*)"
| rex field=_raw mode=sed "s/(\"|\\\\n)//g"
| extract pairdelim="," kvdelim=":"
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @kranthimutyala,

this seems to be a json log, did you tried using spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/spath)?

Ciao.

Giuseppe

0 Karma

kranthimutyala
Path Finder

Hi @gcusello I tried Spath to extract them but unsuccessful. 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

 

| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG

 

Your sample data now gives

LOGLogDateLogIdLogLevelLogMessageLogNoLogTypeMachineNameProcessNameRPAEnvironmentTaskNameTimeTaken 
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }09/23/2022 16:11:13.932002308900.20250824210419999INFOSplunk anypoint log99Process Levelabc-xyz-wdAnypointSplunkTestDEVAnypointTest0If:00:12.628name.first
If Splunk doesn't give you event field, apply spath first to extract event.

 

| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG​

 

 

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @kranthimutyala,

it's strange because it seems to be a json format.

Anyway, in this case you have some regex extraction like the following:

\"LogDate\":\s+\"(?<LogDate>[^\"]+)

that you can test at https://regex101.com/r/IzcMqn/1 

and that you can replicate for all your fields.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...