Splunk Search

Realtime searches (efficiency & results)

troywollenslege
Path Finder

As far as efficiency, we were told that realtime searches take "a fraction" of a CPU core per search. Does it matter if someone is doing realtime-all time, or realtime-5min/30 min window?

I was looking at http://www.splunk.com/view/real-time-in-splunk/SP-CAAAFD7 but wasn’t clear.

Just confirming, if you do a rt30 minute search and you look for an event that follows another event and those events are >30 minutes apart, you wouldn’t see anything.

Also, if you do realtime search for 30 minute window and events come into Splunk with different time greater than 30 minutes (e.g. timezone or bad CPU clock time) you won't see those as either.

Thanks

Tags (1)
0 Karma

iTechEvent
Explorer

Windowed Real Time: uses earliesttime=rt-1 latesttime=rt
Non Windowed Real Time: earliesttime=rt latesttime=rt

Let me share my experience so far on this matter:
They can be compared with these parameters:
1.cpu 2.ram 3. diskspace 4. #events 5. querytime 6. overhead

Windowed RT
cpu-fraction of core per search as above
events-Query returns mostly fixed number of events, with some marginal fluctuation
querytime-mostly the same due to event count
ram-mostly fixed amount with some fluctuations due to above events count
diskspace for query-increasing and can exhaust the disk space quota per user
overheads-Window management overhead

Non Window RT
cpu-fraction of core per search as above
events-Query returns continuously increasing number of events since its all real time and the events continue to increase over time
querytime- as the event counts increases, the query run time also increase due to processing more events
ram-increasing amount of ram consumed as the event count keeps increasing
diskspace for query- same as windowed,increasing and can exhaust the disk space quota per user
overheads-No window management overhead, uses all events in real time

In comparison, Windowed RT preferable even though there is rolling window management overheads due to above plus points. The only minus point is the disk space keeps on increasing and can exhaust the quota. Hence periodically the windowed real time query can be disabled and enabled to clean up the disk space used and start over.

0 Karma

dglinder
Path Finder

bump I hate adding a "me too" for a response...

0 Karma

phoenixdigital
Builder

I would like to know the answer to this as well if anyone knows.

0 Karma
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...