Splunk Cloud Platform

How to set up an automatic way - sourcetype to format a json record

jason_tqb
Explorer

Hello guys.
This is my first post here to ask for help with extracting fields from a JSON object. Below is an example of the record:
{"pod":"fmd9p","time":"2023-10-03T21:49:39.31255352Z", "source":"/var/log/containers/fmd9p_default.log","container_id":"1ae53e1be","log": "I1003 14:49:39.312453 test_main.cc:149] trace_id=\"8aeb0\" event=\"Worker.Finish\" program_run_sec=25.1377 status=\"OK\""}

How can I extract trace_id, event, program_run_sec, and status from the log section automatically by setting up a sourcetype? Is it doable?

Thanks for any help and advise

Labels (1)
0 Karma
1 Solution

bowesmana
SplunkTrust
SplunkTrust

suorce key is the field name, change _raw to log. You don't need the format, as you have specified the field names in the extraction string

Note that the existing JSON needs to be auto extracted, which means that it has to have been set up to do so. It's easy to see just do index=x and look down the left hand side of the display in verbose mode to see if the 'log' field is shown as a field.

View solution in original post

jason_tqb
Explorer

Screenshot 2023-10-03 at 10.45.01 PM.png

Thank you so much for the quick response!
I found this Field transformations to be added in our Splunk cloud. 
Where can I specify the source field log and what should be configured in the Format option there?

Best,

0 Karma

inventsekar
SplunkTrust
SplunkTrust

>>> This is my first post here to ask for help

Welcome to Splunk Community. here you can find almost 90 % of everybody's splunk issues will be solved, your issue is definitely one inside that 90%. 



>>> How can I extract trace_id, event, program_run_sec, and status from the log section automatically by setting up a sourcetype? Is it doable?

it is very much doable. as updated by bowesmana , you should just update one or two config files. thats all. we will guide you on this task step by step. please update us more details on which stage/step you are currently into. 

As u r a new member, i thought to tell you, upvotes / karma points are appreciated by everyone. kindly help those who help you with your karma points, thanks. 

jason_tqb
Explorer

Screenshot 2023-10-03 at 10.50.00 PM.png

 I'm new here and still learning to make the change.
Currently I'm on the Splunk cloud version and this Field transformations is where I can find to add the transform but not sure how I can specify the log field and the Format option there. Should I update the Source Key there?

Thanks for the help!

0 Karma

bowesmana
SplunkTrust
SplunkTrust

You want to create a transforms entry that has something along the lines of

trace_id=\\\\\"(?<trace_id>\w+)\\\\\" event=\\\\\"(?<event>[^\\\\\"]*)\\\\\" program_run_sec=(?<program_run_sec>[\d\.]*) status=\\\\\"(?<status>\w+)

using the source field log, which must have already been extracted. Then create a field extraction using that transforms entry against the sourcetype for this data.

Then you will get the 4 fields wanted automatically in your data.

jason_tqb
Explorer

Thank you so much for the quick response!
I found this Field transformations to be added in our Splunk cloud. 
Where can I specify the source field log and what should be configured in the Format option there?

Best,


Screenshot 2023-10-03 at 10.48.08 PM.png

0 Karma

bowesmana
SplunkTrust
SplunkTrust

suorce key is the field name, change _raw to log. You don't need the format, as you have specified the field names in the extraction string

Note that the existing JSON needs to be auto extracted, which means that it has to have been set up to do so. It's easy to see just do index=x and look down the left hand side of the display in verbose mode to see if the 'log' field is shown as a field.

jason_tqb
Explorer

Wow! it works like a charm! Thank you so much for the help!

Best,

0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...