Splunk Search

Extract Key=value before indexing , and index only the extracted key/values

aelnaggar
Engager

I have data input which returns key=value delimited with space, so I don't need to index all of them , so how can I index some of them and avoid the others..
Also I want when I search for something only the important key/values to show

0 Karma

sc0tt
Builder

I recently filtered unwanted data at index time by using the filter and route method. I think this will allow you to accomplish what you need.

0 Karma

aelnaggar
Engager

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?

0 Karma

aelnaggar
Engager

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?

0 Karma

kristian_kolb
Ultra Champion

Permanently removing (parts of) event data prior to indexing can be done by means of index-time transformations or SEDCMD, read more here;

http://docs.splunk.com/Documentation/Splunk/6.0/Data/Anonymizedatausingconfigurationfiles

The definition of 'important' is hard for anyone but you to make. But changing the search mode might be what you're after;

http://docs.splunk.com/Documentation/Splunk/6.0/Search/Changethesearchmode

This will control how fields will be extracted, if at all. You can probably do this in a more manual fashion, by setting KV_MODE=none for your sourcetype, and making explicit EXTRACTs;

http://docs.splunk.com/Documentation/Splunk/6.0/Knowledge/Createandmaintainsearch-timefieldextractio...

http://docs.splunk.com/Documentation/Splunk/6.0/admin/Propsconf

/k

kristian_kolb
Ultra Champion

The first link in my answer above, demonstrate a few options for removing unwanted data from within each event prior to indexing.

The link provided by @sc0tt in his answer shows how to discard/keep whole events based on individual event content.

0 Karma

aelnaggar
Engager

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?

0 Karma
Get Updates on the Splunk Community!

Splunk Answers Content Calendar, June Edition

Get ready for this week’s post dedicated to Splunk Dashboards! We're celebrating the power of community by ...

What You Read The Most: Splunk Lantern’s Most Popular Articles!

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

See your relevant APM services, dashboards, and alerts in one place with the updated ...

As a Splunk Observability user, you have a lot of data you have to manage, prioritize, and troubleshoot on a ...