Hi,
I'm configuring some new roles, and came across the "schedule_rtsearch" capability. The doc simply says "Lets the user schedule real-time saved searches." What is a scheduled rtsearch? Almost seems like an oxymoron.
In Splunk when you schedule a search you are provided with the option of scheduling a "Report" or an "Alert". The Alert gives you additional options to take some action (ie: send an email or run a script) when a trigger condition is met (ie: the search returns a count greater than 0). A scheduled rtsearch is really an alert which runs continuously realtime so a cron_schedule is irrelevant in this case.
When you create an alert in Splunk through the UI you set the alert type as either "Scheduled" or "Real-time". When you select Real-time, the scheduler will delegate the search and keep that search running continuously. This is a scheduled rtsearch.
You will see the sid of the search in resource_usage.log as : rt_scheduled....
dispatch.earliest_time
defines how far back the rt searches looks over the data as it is running continuously so this is a sliding window. This can be configured in advanced settings of the alert or in the UI when you edit the alert under the trigger conditions > "in" x minutes/hours..etc field refers to the dispatch.earliest_time
"scheduling" it just means that if the node it is running on goes down/restarted or the search gets terminated, the scheduler will make sure it gets delegated to another member (if SHC) or respawned (if standalone SH) and makes sure the search is up and running again so the user does not need to intervene.
This is different than a real-time search in the traditional ad-hoc sense which get's killed when the user stops the search or closes the browser.
The Splunk Monitor Console provides a view of the search activity if you need to determine if a scheduled rtsearch is running:
from MC: >
Search>Activity>Search Activity: Deployment
Panel: Search Activity by Instance
Real-time search: http://docs.splunk.com/Splexicon:Realtimesearch
Real-time schedule search (report or alert): https://docs.splunk.com/Splexicon:Realtimealert and http://docs.splunk.com/Documentation/Splunk/7.0.3/Search/Aboutrealtimesearches
Do not let ANYBODY have this capability, unless your product specifically (like ITSI) needs it. It is the best way to crush your Search Head.
apologies @woodcock i am too late here but would it work with a role not having rtsearch capability ?
doc says
schedule_rtsearch | Lets the user schedule real-time saved searches. The schedule_search and rtsearch capabilities must also be assigned to the role. |
Thanks. But, if you are scheduling a real-time search, how does that work? I would think that it would never end.
Essentially correct. I may be wrong but I always understood it to mean that it will start as per its scheduling and then run real-time until it's manually killed (I guess).
I think the bottom line is that you were correct in noticing it as an odd one and I usually don't allow any real time (rt) capabilities because they are rarely needed and can be so impactfull.