Knowledge Management

Using report-acceleartion/summary-indexing for searches on extracted KPIs

bjoernjensen
Contributor

Hi there!

I am trying to extract certain values (KPIs) into a separated 'area' (now trying a summary index) to be able to do quick searches on them. The daily indexing amount is huge and comes from only 3 different sources. That's why I would like to have a seperated searchable area.

To do my evaluation I am right now using some chatty source: WinEventLog:Security. One Event would look like that:

04/23/2013 03:13:02 PM
LogName=Security
SourceName=Microsoft Windows-Sicherheitsüberprüfung.
EventCode=5152
EventType=0
Type=Informationen
ComputerName=falkbpmtest.falkland.de
TaskCategory=Filterplattform: Verworfene Pakete
OpCode=Info
RecordNumber=39054239
Keywords=Überwachung gescheitert
Message=Von der Windows-Filterplattform wurde ein Paket blockiert.

Anwendungsinformationen:
Prozess-ID: 0
Anwendungsname: -

Netzwerkinformationen:
Richtung: Eingehend
Quelladresse: 192.168.110.82
Quellanschluss: 59433
Zieladresse: 192.168.111.255
Zielanschluss: 1947
Protokoll: 17

Filterinformationen:
Laufzeit-ID des Filters: 195071
Ebenenname: Transport
Laufzeit-ID der Ebene: 13

My test use case: Extract "Quelladresse" and "Quellanschluss" into that named area to be able to do quick searches with those. Having in mind that later those values would be my KPIs.

So I set up two searches which I saved with the option to be accelerated. The search runs within a timerange of "-24h@h" till "now". Summary Range is set to "all time" (on purpose):
as1: host="win12" sourcetype="WinEventLog:Security" | table Quelladresse
as2: host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss

The source is big enough (>10k Events) so splunk actually accelerates these two searches successfully. Typing this exact searches does what I want: accelerated searches. Now I thought I could do something like this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

But this won't get accelerated 😞 Does anyone have any suggestions. I think the challenge here is that I'm not interested in the KPIs alone, but rather want to do very quick searches on them.

Thanks for any suggestions / approaches.

Cheerz - Björn

0 Karma
1 Solution

jtrucks
Splunk Employee
Splunk Employee

You likely want to manually insert the data into a summary index using collect. This way you have an index you can perform a variety of arbitrary searches against without hitting the whole dataset. This is exactly how I've solved the same problem.

So, instead of accelerating the search, have the searches do this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse | collect index=mysummaryindex
host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss | collect index=mysummaryindex

Then run your later search like:

index=mysummaryindex host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

See if that works for you.

View solution in original post

jtrucks
Splunk Employee
Splunk Employee

You likely want to manually insert the data into a summary index using collect. This way you have an index you can perform a variety of arbitrary searches against without hitting the whole dataset. This is exactly how I've solved the same problem.

So, instead of accelerating the search, have the searches do this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse | collect index=mysummaryindex
host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss | collect index=mysummaryindex

Then run your later search like:

index=mysummaryindex host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

See if that works for you.

bjoernjensen
Contributor

Thanks for the answer! Works very good.

After having a closer look how accelerated searches are working I have a much better understanding what their 'goal' is.

0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...