Help with extracting JSON fields

[kranthimutyala]

Hi Team,
I have the event in the below format and want to extract the key-value pairs as fields.

Please help extract fields from LogDate till the user.Thanks

{ [-]
   event: INFO  2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-ust-email-notification-v1-uw-qa].get:\ping:Router.CPU_LITE @6c1fb7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {
  "LogDate": "09/23/2022 16:11:13.932",
  "LogNo": "99",
  "LogLevel": "INFO",
  "LogType": "Process Level",
  "LogMessage": "Splunk anypoint log",
  "TimeTaken": "0:00:12.628",
  "ProcessName": "AnypointSplunkTest",
  "TaskName": "AnypointTest",
  "RPAEnvironment": "DEV",
  "LogId": "002308900.20250824210419999",
  "MachineName": "abc-xyz-efg",
  "User": "name.first"
}
   metaData: { [+]
   }
}

and this is the raw text 

{"metaData":{"sourceApiVersion":"1.0.0-SNAPSHOT","index":"aas","sourceApi":"papi-cust-email-notification-v1-uw-qa","cloudhubEnvironment":"AUTOMATION-QA","tags":""},"event":"INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {\n \"LogDate\": \"09/23/2022 16:11:13.932\",\n \"LogNo\": \"99\",\n \"LogLevel\": \"INFO\",\n \"LogType\": \"Process Level\",\n \"LogMessage\": \"Splunk anypoint log\",\n \"TimeTaken\": \"0:00:12.628\",\n \"ProcessName\": \"AnypointSplunkTest\",\n \"TaskName\": \"AnypointTest\",\n \"RPAEnvironment\": \"DEV\",\n \"LogId\": \"002308900.20250824210419999\",\n \"MachineName\": \"abc-xyz-wd\",\n \"User\": \"name.first\"\n}"}

[johnhuang]

This should work:

| rex "(?<_raw>\"LogDate[^\}]*)"
| rex field=_raw mode=sed "s/(\"|\\\\n)//g"
| extract pairdelim="," kvdelim=":"

[gcusello]

Hi @kranthimutyala,

this seems to be a json log, did you tried using spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/spath)?

Ciao.

Giuseppe

[kranthimutyala]

Hi @gcusello I tried Spath to extract them but unsuccessful. 

[yuanliu · Accepted solution]

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG

Your sample data now gives

LOG

LogDate

LogId

LogLevel

LogMessage

LogNo

LogType

MachineName

ProcessName

RPAEnvironment

TaskName

TimeTaken

{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

09/23/2022 16:11:13.932

002308900.20250824210419999

INFO

Splunk anypoint log

99

Process Level

abc-xyz-wd

AnypointSplunkTest

DEV

AnypointTest

0If:00:12.628

name.first

If Splunk doesn't give you event field, apply spath first to extract event.

| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG​

[gcusello]

Hi @kranthimutyala,

it's strange because it seems to be a json format.

Anyway, in this case you have some regex extraction like the following:

\"LogDate\":\s+\"(?<LogDate>[^\"]+)

that you can test at https://regex101.com/r/IzcMqn/1 

and that you can replicate for all your fields.

Ciao.

Giuseppe