topic How to have Splunk search to include only events outside regular business hours? in Splunk Search

How to have Splunk search to include only events outside regular business hours?

JLopez — Mon, 10 Apr 2023 16:44:12 GMT

Hi Splunkers,

I want to create a search that send results to an "On call" system only for out of hours during monday to Friday from 5:30PM until the next day at 8:30AM and also 24h during the weekend starting on Friday at 5:30PM until Monday at 8:30AM. so basically I don't want to send any results during bussiness hours from 8:30AM till 5:30PM Mon-Friday.

I am not sure if it's easier to set this up using cron time scheduler when I have my search ready or using earliest and latest and some eval command within the search.

Also wondering if this can be achieve within 1 search or should I create 1 for monday to friday and another one for the weekend given that the time ranges are different?

Could Anyone have an idea how to best achieve this?

Much appreciate it.

Re: Splunk search to include only events outside regular business hours.

woodcock — Thu, 06 Apr 2023 18:43:04 GMT

Create a super-set cron covering of all of the times and then add logic to your SPL to short-circuit your search so that it errors on those times that aren't supposed to run. See my unaccepted answer here (UpVotes appreciated):

https://answers.splunk.com/answers/172541/is-it-possible-to-purposely-cause-a-scheduled-sear.html

Be aware that TZ is important here and depends on the Personal Settings of the user used to run the search!!!

Re: Splunk search to include only events outside regular business hours.

PickleRick — Thu, 06 Apr 2023 21:58:55 GMT

Use a subsearch to dynamically generate pair of earliest/latest values. You can then conditionally generate time bounds for your search.

Example:

| tstats earliest_time(source) latest_time(source) where index=_internal
[ | makeresults 
| eval earliest=now(),latest=now()
| eval earliest=if(earliest>2,earliest-100,earliest) 
| eval search="earliest=".earliest." latest=".latest
|table search]

This subsearch initially generates earlies=latest=now(), which would yield no results at all. Then conditionally (the condition is always true of course in this case) rolls the earliest limit 100 seconds into the past so the search range for tstats effectively becomes "100s ago till now".

Re: Splunk search to include only events outside regular business hours.

gcusello — Fri, 07 Apr 2023 05:56:17 GMT

Hi @JLopez,

if you need to filter events outside the office hours, the solution from @PickleRick or @woodcock are the solution to your requirement.

If instead you want also manage holydays, you need a more complex solution.

In this case you need to create a lookup containing the holydays (called e.g. CAL) containing at least a column (called e.g. Holiday).

then you should run a search like the following:

| from datamodel:"Authentication" | eval Day=strftime(_time,"%d/%m/%Y"), WeekDay=strftime(_time,"%A"), Hours=strftime(_time,"%H") | search [ | inputlookup SIEMCAL.csv | fields Day ] OR WeekDay="Saturday" OR Weekday="Sunday" OR Hours>18 OR Hours<8 | table _time user host

Putting attention that the format of the Day column in the lookup is the same of the Day variable.

If you haven't the Authenticatin Datamodel, is more complex because you have to identify Login or Logfail or Logout events for all your platforms.

In addition you could put the subsearch for non working hours in a macro (called e.g. "NotWorkingTime") that you can call every time you need.

| eval Day=strftime(_time,"%d/%m/%Y"), WeekDay=strftime(_time,"%A"), Hours=strftime(_time,"%H") | search [ | inputlookup SIEMCAL.csv | fields Day ] OR WeekDay="Saturday" OR Weekday="Sunday" OR Hours>18 OR Hours<8

Ciao.

Giuseppe

Re: Splunk search to include only events outside regular business hours.

gcusello — Fri, 14 Apr 2023 15:39:24 GMT

hi @JLopez ,

good for you, see next time!

Ciao and happy splunking

Giuseppe

P.S.: Karma Points are appreciated by all the contributors 😉