Extract Key=value before indexing , and index only the extracted key/values

[aelnaggar]

I have data input which returns key=value delimited with space, so I don't need to index all of them , so how can I index some of them and avoid the others..
Also I want when I search for something only the important key/values to show

[sc0tt]

I recently filtered unwanted data at index time by using the filter and route method. I think this will allow you to accomplish what you need.

[aelnaggar]

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?

[aelnaggar]

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?

[kristian_kolb]

Permanently removing (parts of) event data prior to indexing can be done by means of index-time transformations or SEDCMD, read more here;

http://docs.splunk.com/Documentation/Splunk/6.0/Data/Anonymizedatausingconfigurationfiles

The definition of 'important' is hard for anyone but you to make. But changing the search mode might be what you're after;

http://docs.splunk.com/Documentation/Splunk/6.0/Search/Changethesearchmode

This will control how fields will be extracted, if at all. You can probably do this in a more manual fashion, by setting KV_MODE=none for your sourcetype, and making explicit EXTRACTs;

http://docs.splunk.com/Documentation/Splunk/6.0/Knowledge/Createandmaintainsearch-timefieldextractio...

http://docs.splunk.com/Documentation/Splunk/6.0/admin/Propsconf

/k

[kristian_kolb]

The first link in my answer above, demonstrate a few options for removing unwanted data from within each event prior to indexing.

The link provided by @sc0tt in his answer shows how to discard/keep whole events based on individual event content.

[aelnaggar]

Hi Kristian, thanks for your answer , the main goal here is to exclude unneeded data from being stored in splunk, so I needed to store only the important one to me..

so thinking about how to do this , and extract some parts of the incoming messages to Splunk store it and indexing it..

still your answer valid in that case ?