Splunk Search

create scheduled search and alerting between 7pm to 7am

ash2
Explorer

How to schedule search between 7pm to 7am and alert if and only if there is an event recorded between 7pm to 7am? my cron expression is */15 19-23,0-6 * * *. What should be the earliest and latest value?

Labels (1)
0 Karma
1 Solution

richgalloway
SplunkTrust
SplunkTrust

You only need to report if an event arrived since the last time the search ran.  If an event came in earlier then the previous run of the search would have found it.  So, run every 15 minutes and use earliest=-15m or run once at 7am and use earliest=-12h or something in between.

---
If this reply helps you, Karma would be appreciated.

View solution in original post

richgalloway
SplunkTrust
SplunkTrust

If the search runs every 15 minutes then there's little reason to search more than 20 minute back.  So, earliest=-20m latest=now.  What is the use case?

---
If this reply helps you, Karma would be appreciated.
0 Karma

ash2
Explorer

@richgalloway thank you for your reply. so, what im trying to achieve is, i want to trigger an email alert if there is any event between the time period 7pm to next day 7am. I'm using scheduled alerting mechanism. My cron scheduler runs every 15mins starting from 7pm until 7am next day. During this period if it comes across any event record after 7pm and before 7am next day from a search. I want to trigger an email. But im struggling to embed time range for search between 7pm to 7am. 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

You only need to report if an event arrived since the last time the search ran.  If an event came in earlier then the previous run of the search would have found it.  So, run every 15 minutes and use earliest=-15m or run once at 7am and use earliest=-12h or something in between.

---
If this reply helps you, Karma would be appreciated.

PickleRick
SplunkTrust
SplunkTrust

There is a possible use case of searching throughout the whole 7pm-7am range if there is a possibility of an event indexing late (with a significant lag). While typically it signifies problems with data quality or problems with the processing pipeline, there are some ingestion schemes for which that can be a normal mode of operation (for example WEF in pull mode has 30minutes interval by default if I remember correctly).

In such case you can manipulate your time range similarily to

earliest=@d+19h

You should even be able to do (but I haven't tested it since I don't have a Splunk instance available at the moment) something like

earliest=-12h@d+19h

Fiddle with this and check if it's what you need

But if your data is ingested with a constant flow then you should be ok with monitoring just most recently ingested part as @richgalloway said. Either use a searching window slightly longer than your scheduled interval in order not to miss any slightly lagged events or use continuous schedule.

Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...