Splunk Search

Help with extracting JSON fields

kranthimutyala
Path Finder

Hi Team,
I have the event in the below format and want to extract the key-value pairs as fields.

Please help extract fields from LogDate till the user.Thanks

 

 

{ [-]
   event: INFO  2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-ust-email-notification-v1-uw-qa].get:\ping:Router.CPU_LITE @6c1fb7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {
  "LogDate": "09/23/2022 16:11:13.932",
  "LogNo": "99",
  "LogLevel": "INFO",
  "LogType": "Process Level",
  "LogMessage": "Splunk anypoint log",
  "TimeTaken": "0:00:12.628",
  "ProcessName": "AnypointSplunkTest",
  "TaskName": "AnypointTest",
  "RPAEnvironment": "DEV",
  "LogId": "002308900.20250824210419999",
  "MachineName": "abc-xyz-efg",
  "User": "name.first"
}
   metaData: { [+]
   }
}

 

 

 

and this is the raw text 

{"metaData":{"sourceApiVersion":"1.0.0-SNAPSHOT","index":"aas","sourceApi":"papi-cust-email-notification-v1-uw-qa","cloudhubEnvironment":"AUTOMATION-QA","tags":""},"event":"INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {\n \"LogDate\": \"09/23/2022 16:11:13.932\",\n \"LogNo\": \"99\",\n \"LogLevel\": \"INFO\",\n \"LogType\": \"Process Level\",\n \"LogMessage\": \"Splunk anypoint log\",\n \"TimeTaken\": \"0:00:12.628\",\n \"ProcessName\": \"AnypointSplunkTest\",\n \"TaskName\": \"AnypointTest\",\n \"RPAEnvironment\": \"DEV\",\n \"LogId\": \"002308900.20250824210419999\",\n \"MachineName\": \"abc-xyz-wd\",\n \"User\": \"name.first\"\n}"}

Labels (3)
Tags (3)
0 Karma
1 Solution

yuanliu
SplunkTrust
SplunkTrust

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

 

| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG

 

Your sample data now gives

LOGLogDateLogIdLogLevelLogMessageLogNoLogTypeMachineNameProcessNameRPAEnvironmentTaskNameTimeTaken 
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }09/23/2022 16:11:13.932002308900.20250824210419999INFOSplunk anypoint log99Process Levelabc-xyz-wdAnypointSplunkTestDEVAnypointTest0If:00:12.628name.first
If Splunk doesn't give you event field, apply spath first to extract event.

 

| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG​

 

 

View solution in original post

Tags (1)
0 Karma

johnhuang
Motivator

This should work:

| rex "(?<_raw>\"LogDate[^\}]*)"
| rex field=_raw mode=sed "s/(\"|\\\\n)//g"
| extract pairdelim="," kvdelim=":"
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @kranthimutyala,

this seems to be a json log, did you tried using spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/spath)?

Ciao.

Giuseppe

0 Karma

kranthimutyala
Path Finder

Hi @gcusello I tried Spath to extract them but unsuccessful. 

0 Karma

yuanliu
SplunkTrust
SplunkTrust

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

 

| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG

 

Your sample data now gives

LOGLogDateLogIdLogLevelLogMessageLogNoLogTypeMachineNameProcessNameRPAEnvironmentTaskNameTimeTaken 
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }09/23/2022 16:11:13.932002308900.20250824210419999INFOSplunk anypoint log99Process Levelabc-xyz-wdAnypointSplunkTestDEVAnypointTest0If:00:12.628name.first
If Splunk doesn't give you event field, apply spath first to extract event.

 

| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG​

 

 

Tags (1)
0 Karma

gcusello
SplunkTrust
SplunkTrust

Hi @kranthimutyala,

it's strange because it seems to be a json format.

Anyway, in this case you have some regex extraction like the following:

\"LogDate\":\s+\"(?<LogDate>[^\"]+)

that you can test at https://regex101.com/r/IzcMqn/1 

and that you can replicate for all your fields.

Ciao.

Giuseppe

0 Karma
Get Updates on the Splunk Community!

Observability Unlocked: Kubernetes Monitoring with Splunk Observability Cloud

  Ready to master Kubernetes and cloud monitoring like the pros?Join Splunk’s Growth Engineering team for an ...

Wrapping Up Cybersecurity Awareness Month

October might be wrapping up, but for Splunk Education, cybersecurity awareness never goes out of season. ...

🌟 From Audit Chaos to Clarity: Welcoming Audit Trail v2

&#x1f5e3; You Spoke, We Listened  Audit Trail v2 wasn’t written in isolation—it was shaped by your voices.  In ...