topic Set up splunk Alert - a complicated one in Alerting

Set up splunk Alert - a complicated one

loveforsplunk — Sun, 21 May 2017 04:45:06 GMT

Here is my log file having a key word "error":

My search is : index=abc host="123" "error" source="efg/*"

My search returns results as below (check out the timings ) , suppose below is the event section:

Time Event
5/20/17 1:00:45.000 AM Completed at Sat May 20 03:00:45 2017

Under the _time section , the time which is displayed is 2hrs less than the time that is displayed in the logs(as you can see from the event section).

Now , suppose there is a failure in my log which I came to know right now. I go to splunk and check I do not see any result for the last 15 minutes even but when I do last 2 hours , I get to see the result.

Please tell me how do i set this alert. if I am setting to check every 5 minutes, I do not get any alert . When I did -2h as start time and now as finish time , I still did not get any alert . Now I did it -2h@h which I am sure will work or not until there is a failure.

Also, I have selected Run every minute while setting the alert.

Re: Set up splunk Alert - a complicated one

dineshraj9 — Sun, 21 May 2017 06:17:14 GMT

You need to fix the timezone of your logs first. The 2hr difference is because of the logs being set to incorrect timezone.

https://docs.splunk.com/Documentation/Splunk/6.6.0/Data/Applytimezoneoffsetstotimestamps

I suggest you get the TZ attribute in props.conf for the sourcetype.
This will fix the timezone issue and logs will appear real time.

Re: Set up splunk Alert - a complicated one

loveforsplunk — Sun, 21 May 2017 15:10:18 GMT

But for the same server , I have alert set up for other logs as well. For those I get the expected output . Only for this particular log I do not get, not sure why.

Do u mean , there is a way to set up timezone for particular logs ??

In my Splunk user settings , my timezone set is correct and the logs timezone in its server as I see is also the same as mine , then why do I get something else in _time ?

Re: Set up splunk Alert - a complicated one

adonio — Mon, 22 May 2017 00:14:22 GMT

it has nothing to do with your users timezone,
check the link in the answer

Re: Set up splunk Alert - a complicated one

dineshraj9 — Mon, 22 May 2017 05:07:38 GMT

Yes, you can setup for timezone for each log in different ways.

In props.conf, you can set TZ attribute for the particular sourcetype.

In inputs.conf, you can set _tzhint field for the particular log monitor stanza.

Re: Set up splunk Alert - a complicated one

loveforsplunk — Mon, 22 May 2017 16:49:06 GMT

oh ok. I will work on this. Thank you so much Dinesh.