Getting Data In

Extract fields from RFC5424 syslog with nested json field

splunkreal
Motivator

Hello, I put this regex on SHC inline extraction : 

"<(?<pri>\d+)>1\s(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?[+-]\d{2}:\d{2})\s(?<hostname>[^\s]+)\s(?<appname>[^\s]+)\s(?<procid>[^\s]+)\s(?<msgid>[^\s]+)\s(?<structured_data>\S+)\s(?<json_msg>\{.*\})"


however json_msg needs | spath input=json_msg

Is it possible to auto extract fields contained in json_msg to avoid adding | spath input=json_msg at search time?

Thanks. 

* If this helps, please upvote or accept solution if it solved *
Labels (4)
0 Karma

PickleRick
SplunkTrust
SplunkTrust

Not at this time. Splunk can auto-extract values only if the whole _raw message consists of the structured data blob. There is an open idea on ideas.splunk.com - https://ideas.splunk.com/ideas/EID-I-208

It is marked as future prospect but of course voting on this issue might provide some additional push.

The alternative would be to cut the remainder of the event so that only the json part is left but this way you're losing some data.

tej57
Builder

Hello @splunkreal,

If you're using rex to extract the fields at search time, there's no way that Splunk will auto extract the fields that are part of your json_msg field. 

However, you can write the regex and have the fields extracted at search time using the field extraction from Settings -> Fields -> Field Extraction and define it under the sourcetype. So that everytime you run index based search, the fields json_msg will be extracted automatically and then you can use | spath json_msg to extract the subsequent fields. 

Alternatively, from the source if you are able to convert the whole string into JSON format, the nested json fields will be extracted automatically.

Regards,
Tejas.

---

If the above solution helps, an upvote is appreciated..!!

Get Updates on the Splunk Community!

.conf25 Community Recap

Hello Splunkers, And just like that, .conf25 is in the books! What an incredible few days — full of learning, ...

Splunk App Developers | .conf25 Recap & What’s Next

If you stopped by the Builder Bar at .conf25 this year, thank you! The retro tech beer garden vibes were ...

Congratulations to the 2025-2026 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...