Splunk Enterprise

How to enable Splunk to set the time of an event based on a condition during indexation?

manikanta461
Explorer

My event will be as follows:

#2020-01-01;12:00:00#2020-01-01;12:00:00#content on the event. 

#2020-01-01;12:00:01#1970-01-01;00:00:00#content on the event. 

I have configured my sourcetype to pick the time highlighted in orange as the time of the event during indexation; however, sometimes in our logs we have the time recorded with a date starting with 1970, Splunk doesn't recognize it because of max days ago limit. In such cases, is there a possibility to add a condition to fetch the date in Blue as the date of the event?

P.S: I can change the _time of the event in search head but, I'm trying for a solution that will index the events directly with Blue time as the time of the event, when there is 1970 in orange time.

0 Karma

niketnilay
Legend

@manikanta461 assuming whenever you get 1970 date it is always1970-01-01;00:00:00. You can use SEDCMD through props.conf or REGEX through transforms.conf to find starting date and hardcoded 1970-01-01;00:00:00. Replace whenever hardcoded 1970-01-01;00:00:00 is found.

Something like the following which captures 1st group as first date value and 2nd group as 1970-01-01;00:00:00. All you have to do is replace \2 with \1

"s/^(#[^#]+)(\#1970-01-01;00:00:00)/\1\1/g"

Refer to the anonymization documentation which in your case you can use for replacement of capturing group 1 instead of 2 whenever hardcoded 1970 date is found.

https://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedata#Anonymize_data_with_a_sed_scr...

Following is a run anywhere example to demo how this works. Second row has 1970 event as per your question.

 

| makeresults 
| eval _raw="#2020-01-01;12:00:00#2020-01-01;12:00:00#content on the event1." 
| append 
    [| makeresults 
    | eval _raw="#2020-01-01;12:00:00#1970-01-01;00:00:00#content on the event1."] 
| rex mode=sed "s/^(#[^#]+)(\#1970-01-01;00:00:00)/\1\1/g"

 

____________________________________________
| makeresults | eval message= "Happy Splunking!!!"
0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!