create scheduled search and alerting between 7pm to 7am

[ash2]

How to schedule search between 7pm to 7am and alert if and only if there is an event recorded between 7pm to 7am? my cron expression is */15 19-23,0-6 * * *. What should be the earliest and latest value?

[richgalloway]

If the search runs every 15 minutes then there's little reason to search more than 20 minute back.  So, earliest=-20m latest=now.  What is the use case?

[ash2]

@richgalloway thank you for your reply. so, what im trying to achieve is, i want to trigger an email alert if there is any event between the time period 7pm to next day 7am. I'm using scheduled alerting mechanism. My cron scheduler runs every 15mins starting from 7pm until 7am next day. During this period if it comes across any event record after 7pm and before 7am next day from a search. I want to trigger an email. But im struggling to embed time range for search between 7pm to 7am. 

[richgalloway · Accepted solution]

You only need to report if an event arrived since the last time the search ran.  If an event came in earlier then the previous run of the search would have found it.  So, run every 15 minutes and use earliest=-15m or run once at 7am and use earliest=-12h or something in between.

[PickleRick]

There is a possible use case of searching throughout the whole 7pm-7am range if there is a possibility of an event indexing late (with a significant lag). While typically it signifies problems with data quality or problems with the processing pipeline, there are some ingestion schemes for which that can be a normal mode of operation (for example WEF in pull mode has 30minutes interval by default if I remember correctly).

In such case you can manipulate your time range similarily to

earliest=@d+19h

You should even be able to do (but I haven't tested it since I don't have a Splunk instance available at the moment) something like

earliest=-12h@d+19h

Fiddle with this and check if it's what you need

But if your data is ingested with a constant flow then you should be ok with monitoring just most recently ingested part as @richgalloway said. Either use a searching window slightly longer than your scheduled interval in order not to miss any slightly lagged events or use continuous schedule.