Hi Team,
I have the event in the below format and want to extract the key-value pairs as fields.
Please help extract fields from LogDate till the user.Thanks
{ [-]
event: INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-ust-email-notification-v1-uw-qa].get:\ping:Router.CPU_LITE @6c1fb7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {
"LogDate": "09/23/2022 16:11:13.932",
"LogNo": "99",
"LogLevel": "INFO",
"LogType": "Process Level",
"LogMessage": "Splunk anypoint log",
"TimeTaken": "0:00:12.628",
"ProcessName": "AnypointSplunkTest",
"TaskName": "AnypointTest",
"RPAEnvironment": "DEV",
"LogId": "002308900.20250824210419999",
"MachineName": "abc-xyz-efg",
"User": "name.first"
}
metaData: { [+]
}
}
and this is the raw text
{"metaData":{"sourceApiVersion":"1.0.0-SNAPSHOT","index":"aas","sourceApi":"papi-cust-email-notification-v1-uw-qa","cloudhubEnvironment":"AUTOMATION-QA","tags":""},"event":"INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: {\n \"LogDate\": \"09/23/2022 16:11:13.932\",\n \"LogNo\": \"99\",\n \"LogLevel\": \"INFO\",\n \"LogType\": \"Process Level\",\n \"LogMessage\": \"Splunk anypoint log\",\n \"TimeTaken\": \"0:00:12.628\",\n \"ProcessName\": \"AnypointSplunkTest\",\n \"TaskName\": \"AnypointTest\",\n \"RPAEnvironment\": \"DEV\",\n \"LogId\": \"002308900.20250824210419999\",\n \"MachineName\": \"abc-xyz-wd\",\n \"User\": \"name.first\"\n}"}
@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)
As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following
INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }
Here, you just need to extract that JSON object, then apply spath.
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG
Your sample data now gives
LOG | LogDate | LogId | LogLevel | LogMessage | LogNo | LogType | MachineName | ProcessName | RPAEnvironment | TaskName | TimeTaken | |
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" } | 09/23/2022 16:11:13.932 | 002308900.20250824210419999 | INFO | Splunk anypoint log | 99 | Process Level | abc-xyz-wd | AnypointSplunkTest | DEV | AnypointTest | 0If:00:12.628 | name.first |
| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG
This should work:
| rex "(?<_raw>\"LogDate[^\}]*)"
| rex field=_raw mode=sed "s/(\"|\\\\n)//g"
| extract pairdelim="," kvdelim=":"
Hi @kranthimutyala,
this seems to be a json log, did you tried using spath command (https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/spath)?
Ciao.
Giuseppe
Hi @gcusello I tried Spath to extract them but unsuccessful.
@kranthimutyala When you say "unsuccessful", you need to illustrate the output and explain why you consider it unsuccessful. (Perhaps you could have explained this in the first problem statement.)
As @gcusello said, your data is compliant JSON, so Splunk should already have given a field "event" - which itself is a combination of free text with an embedded compliant JSON object like the following
INFO 2022-09-23 11:49:59,033 [[MuleRuntime].uber.01: [papi-cust-email-notification-v1-uw2-qa].get:\ping:Router.CPU_LITE @6f3b7] org.mule.runtime.core.internal.processor.LoggerMessageProcessor: { "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" }
Here, you just need to extract that JSON object, then apply spath.
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG
Your sample data now gives
LOG | LogDate | LogId | LogLevel | LogMessage | LogNo | LogType | MachineName | ProcessName | RPAEnvironment | TaskName | TimeTaken | |
{ "LogDate": "09/23/2022 16:11:13.932", "LogNo": "99", "LogLevel": "INFO", "LogType": "Process Level", "LogMessage": "Splunk anypoint log", "TimeTaken": "0:00:12.628", "ProcessName": "AnypointSplunkTest", "TaskName": "AnypointTest", "RPAEnvironment": "DEV", "LogId": "002308900.20250824210419999", "MachineName": "abc-xyz-wd", "User": "name.first" } | 09/23/2022 16:11:13.932 | 002308900.20250824210419999 | INFO | Splunk anypoint log | 99 | Process Level | abc-xyz-wd | AnypointSplunkTest | DEV | AnypointTest | 0If:00:12.628 | name.first |
| spath
| eval LOG = replace(event, "^[^{]+", "")
| spath input=LOG
Hi @kranthimutyala,
it's strange because it seems to be a json format.
Anyway, in this case you have some regex extraction like the following:
\"LogDate\":\s+\"(?<LogDate>[^\"]+)
that you can test at https://regex101.com/r/IzcMqn/1
and that you can replicate for all your fields.
Ciao.
Giuseppe