Archive

Event aggregation before Indexing

Communicator

Hi

we have many sources that sends us a lot of similar events (DNS for example)

a.host.com 1.1.1.1

a.host.com 1.1.1.1

a.host.com 1.1.1.1

Is there a way for me to aggregate the events to one single event and just add the count of events that happened (lets say in one minute) ? so I will only index:

a.host.com 1.1.1.1 count=3

Is it possible to do this in splunk ( maybe with regex in transforms)?

For now we are depended on 3rd party agents like arcsight in front of splunk to do the aggregation - I would really like to see this feature in splunk

Any ideas?

1 Solution

Champion

No you will not be able to do that at all. One possibility is to use a powershell script and count the events and send using scripted input.

View solution in original post

SplunkTrust
SplunkTrust

A) Create a preprocessor program in any language that will aggregate the data and create a single event.

B) Add the events to a temporary index that rolls off in a few hours or days, then create a summary index using that temporary index as input.

0 Karma

Champion

No you will not be able to do that at all. One possibility is to use a powershell script and count the events and send using scripted input.

View solution in original post

Communicator

will do , thanks MuS

0 Karma

SplunkTrust
SplunkTrust

Hi gooza,

feel free to open an enhancement request on the support portal - this would be a P4 support case http://www.splunk.com/support .

cheers, MuS

0 Karma

Communicator

ok ,thanks for the replay, I hope splunk will add it in the future.

if anyone else need this feature please vote up this question so splunk can see the need.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!