Splunk Search

How to set up a NEAR real time search in Splunk 6.6.3

nls7010
Path Finder

I have a client that wants to set up a "near" real time search in Splunk. Can this be done (it needs to be continuous), if so, would we need to use a cron job for searches like this or can we dong something like -5rt to rt? Or would that still be considered a "real time" search? The search needs to be run continuously to catch the errors as they come in.

Tags (1)
0 Karma
1 Solution

arkadyz1
Builder

Anything that ends at "rt" is a real-time search. Ask yourself this: how "continuous" do you want it to be? If you want granularity up to, say, 2 seconds, you can have a bigger base search of -5m to -2s, then add to it with smaller -4s to -2s searches running every 2 seconds.

On forms and dashboards, you can set a refresh time separately on each element or globally at the top level.

View solution in original post

0 Karma

nls7010
Path Finder

Their search is based off the time selection and it's all time. (index="myindex" REMOTE-DEVICE STATUS CHECK is all there is to it. If the events are greater than one they invoke the script in the $HOME/splunkforwarder/bin/scripts directory. They invoke it when the number of events is greater than zero and if it's true, then they alert and throttle the alerting for 60 minutes.

0 Karma

arkadyz1
Builder

Anything that ends at "rt" is a real-time search. Ask yourself this: how "continuous" do you want it to be? If you want granularity up to, say, 2 seconds, you can have a bigger base search of -5m to -2s, then add to it with smaller -4s to -2s searches running every 2 seconds.

On forms and dashboards, you can set a refresh time separately on each element or globally at the top level.

0 Karma

nls7010
Path Finder

The clients are looking for a particular phrase in their logs and want it to be a continuous search. Not certain how I would split this up into the two searches you mentioned above.

0 Karma

arkadyz1
Builder

It seems like they want to react to something quickly. The question is: how quickly? They must have some "reaction time" allowed.

Another question: what are they using to launch the searches? Some kind of Splunk SDK? If yes, any searches, either one shot or regular, have "earliest_time" and "latest_time" keyword attributes that can be added. I did it in Python but I'd assume it's true in any SDK.

If they need it on a dashboard, many elements can have a refresh value. On the top level (form or dashboard tag), there is a refresh attribute which has a numerical value in seconds. A similar attribute can be in many other elements, such as table or chart.

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...