Using report-acceleartion/summary-indexing for searches on extracted KPIs

[bjoernjensen]

Hi there!

I am trying to extract certain values (KPIs) into a separated 'area' (now trying a summary index) to be able to do quick searches on them. The daily indexing amount is huge and comes from only 3 different sources. That's why I would like to have a seperated searchable area.

To do my evaluation I am right now using some chatty source: WinEventLog:Security. One Event would look like that:

04/23/2013 03:13:02 PM
LogName=Security
SourceName=Microsoft Windows-Sicherheitsüberprüfung.
EventCode=5152
EventType=0
Type=Informationen
ComputerName=falkbpmtest.falkland.de
TaskCategory=Filterplattform: Verworfene Pakete
OpCode=Info
RecordNumber=39054239
Keywords=Überwachung gescheitert
Message=Von der Windows-Filterplattform wurde ein Paket blockiert.

Anwendungsinformationen:
Prozess-ID: 0
Anwendungsname: -

Netzwerkinformationen:
Richtung: Eingehend
Quelladresse: 192.168.110.82
Quellanschluss: 59433
Zieladresse: 192.168.111.255
Zielanschluss: 1947
Protokoll: 17

Filterinformationen:
Laufzeit-ID des Filters: 195071
Ebenenname: Transport
Laufzeit-ID der Ebene: 13

My test use case: Extract "Quelladresse" and "Quellanschluss" into that named area to be able to do quick searches with those. Having in mind that later those values would be my KPIs.

So I set up two searches which I saved with the option to be accelerated. The search runs within a timerange of "-24h@h" till "now". Summary Range is set to "all time" (on purpose):
as1: host="win12" sourcetype="WinEventLog:Security" | table Quelladresse
as2: host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss

The source is big enough (>10k Events) so splunk actually accelerates these two searches successfully. Typing this exact searches does what I want: accelerated searches. Now I thought I could do something like this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

But this won't get accelerated 😞 Does anyone have any suggestions. I think the challenge here is that I'm not interested in the KPIs alone, but rather want to do very quick searches on them.

Thanks for any suggestions / approaches.

Cheerz - Björn

[jtrucks · Accepted solution]

You likely want to manually insert the data into a summary index using collect. This way you have an index you can perform a variety of arbitrary searches against without hitting the whole dataset. This is exactly how I've solved the same problem.

So, instead of accelerating the search, have the searches do this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse | collect index=mysummaryindex
host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss | collect index=mysummaryindex

Then run your later search like:

index=mysummaryindex host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

See if that works for you.

[bjoernjensen]

Thanks for the answer! Works very good.

After having a closer look how accelerated searches are working I have a much better understanding what their 'goal' is.