Knowledge Management

Using report-acceleartion/summary-indexing for searches on extracted KPIs

bjoernjensen
Contributor

Hi there!

I am trying to extract certain values (KPIs) into a separated 'area' (now trying a summary index) to be able to do quick searches on them. The daily indexing amount is huge and comes from only 3 different sources. That's why I would like to have a seperated searchable area.

To do my evaluation I am right now using some chatty source: WinEventLog:Security. One Event would look like that:

04/23/2013 03:13:02 PM
LogName=Security
SourceName=Microsoft Windows-Sicherheitsüberprüfung.
EventCode=5152
EventType=0
Type=Informationen
ComputerName=falkbpmtest.falkland.de
TaskCategory=Filterplattform: Verworfene Pakete
OpCode=Info
RecordNumber=39054239
Keywords=Überwachung gescheitert
Message=Von der Windows-Filterplattform wurde ein Paket blockiert.

Anwendungsinformationen:
Prozess-ID: 0
Anwendungsname: -

Netzwerkinformationen:
Richtung: Eingehend
Quelladresse: 192.168.110.82
Quellanschluss: 59433
Zieladresse: 192.168.111.255
Zielanschluss: 1947
Protokoll: 17

Filterinformationen:
Laufzeit-ID des Filters: 195071
Ebenenname: Transport
Laufzeit-ID der Ebene: 13

My test use case: Extract "Quelladresse" and "Quellanschluss" into that named area to be able to do quick searches with those. Having in mind that later those values would be my KPIs.

So I set up two searches which I saved with the option to be accelerated. The search runs within a timerange of "-24h@h" till "now". Summary Range is set to "all time" (on purpose):
as1: host="win12" sourcetype="WinEventLog:Security" | table Quelladresse
as2: host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss

The source is big enough (>10k Events) so splunk actually accelerates these two searches successfully. Typing this exact searches does what I want: accelerated searches. Now I thought I could do something like this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

But this won't get accelerated 😞 Does anyone have any suggestions. I think the challenge here is that I'm not interested in the KPIs alone, but rather want to do very quick searches on them.

Thanks for any suggestions / approaches.

Cheerz - Björn

0 Karma
1 Solution

jtrucks
Splunk Employee
Splunk Employee

You likely want to manually insert the data into a summary index using collect. This way you have an index you can perform a variety of arbitrary searches against without hitting the whole dataset. This is exactly how I've solved the same problem.

So, instead of accelerating the search, have the searches do this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse | collect index=mysummaryindex
host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss | collect index=mysummaryindex

Then run your later search like:

index=mysummaryindex host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

See if that works for you.

--
Jesse Trucks
Minister of Magic

View solution in original post

jtrucks
Splunk Employee
Splunk Employee

You likely want to manually insert the data into a summary index using collect. This way you have an index you can perform a variety of arbitrary searches against without hitting the whole dataset. This is exactly how I've solved the same problem.

So, instead of accelerating the search, have the searches do this:

host="win12" sourcetype="WinEventLog:Security" | table Quelladresse | collect index=mysummaryindex
host="win12" sourcetype="WinEventLog:Security" | table Quellanschluss | collect index=mysummaryindex

Then run your later search like:

index=mysummaryindex host="win12" sourcetype="WinEventLog:Security" | table Quelladresse Quellanschluss

See if that works for you.

--
Jesse Trucks
Minister of Magic

bjoernjensen
Contributor

Thanks for the answer! Works very good.

After having a closer look how accelerated searches are working I have a much better understanding what their 'goal' is.

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...