topic Re: Help with extracting JSON fields in Splunk Search

Help with extracting JSON fields

kranthimutyala — Fri, 23 Sep 2022 12:16:44 GMT

Hi Team,
I have the event in the below format and want to extract the key-value pairs as fields.

Please help extract fields from LogDate till the user.Thanks

{ [-] event: INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-ust-email-notification-v1-uw-qa].get:\ping:Router.CPU_LITE @6c1fb7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-efg", "User": "name.first" } metaData: { [+] } }

and this is the raw text

{"metaData":{"sourceApiVersion":"1.0.0-SNAPSHOT","index":"aas","sourceApi":"papi-cust-email-notification-v1-uw-qa","cloudhubEnvironment":"AUTOMATION-QA","tags":""},"event":"INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {\n \"LogDate\": \"09/23/2022 16:11:13.932\",\n \"LogNo\": \"99\",\n \"LogLevel\": \"INFO\",\n \"LogType\": \"Process Level\",\n \"LogMessage\": \"Splunk anypoint log\",\n \"TimeTaken\": \"0:00:12.628\",\n \"ProcessName\": \"AnypointSplunkTest\",\n \"TaskName\": \"AnypointTest\",\n \"RPAEnvironment\": \"DEV\",\n \"LogId\": \"002308900.20250824210419999\",\n \"MachineName\": \"abc-xyz-wd\",\n \"User\": \"name.first\"\n}"}

Re: Help with extracting JSON fields

gcusello — Fri, 23 Sep 2022 13:58:49 GMT

Hi @kranthimutyala,

this seems to be a json log, did you tried using spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/spath)?

Ciao.

Giuseppe

Re: Help with extracting JSON fields

kranthimutyala — Fri, 23 Sep 2022 14:31:37 GMT

Hi @gcusello I tried Spath to extract them but unsuccessful.

Re: Help with extracting JSON fields

gcusello — Fri, 23 Sep 2022 15:01:05 GMT

Hi @kranthimutyala,

it's strange because it seems to be a json format.

Anyway, in this case you have some regex extraction like the following:

\"LogDate\":\s+\"(?<LogDate>[^\"]+)

that you can test at https://regex101.com/r/IzcMqn/1

and that you can replicate for all your fields.

Ciao.

Giuseppe

Re: Help with extracting JSON fields

yuanliu — Fri, 23 Sep 2022 18:07:32 GMT

@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)

As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following

INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

Here, you just need to extract that JSON object, then apply spath.

| eval LOG = replace(event, "^[^{]+", "") | spath input=LOG

Your sample data now gives

LOG

LogDate

LogId

LogLevel

LogMessage

LogNo

LogType

MachineName

ProcessName

RPAEnvironment

TaskName

TimeTaken

{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }

09/23/2022 16:11:13.932

002308900.20250824210419999

INFO

Splunk anypoint log

Process Level

abc-xyz-wd

AnypointSplunkTest

DEV

AnypointTest

0If:00:12.628

name.first

If Splunk doesn't give you event field, apply spath first to extract event.

| spath | eval LOG = replace(event, "^[^{]+", "") | spath input=LOG

Re: Help with extracting JSON fields

johnhuang — Fri, 23 Sep 2022 18:37:42 GMT

This should work:

| rex "(?<_raw>\"LogDate[^\}]*)" | rex field=_raw mode=sed "s/(\"|\\\\n)//g" | extract pairdelim="," kvdelim=":"