topic Re: Indexing only specific fields in an event in Getting Data In

Indexing only specific fields in an event

pdash — Mon, 18 Feb 2013 22:32:57 GMT

I want to index only specific fields like error status in an event and discard the rest. How do I set splunk to do this?

Re: Indexing only specific fields in an event

Ayn — Mon, 18 Feb 2013 23:41:36 GMT

Splunk has inbuilt functionality by filtering events by sending them to a queue called the nullQueue, which builds on the same concept as /dev/null in UNIX. This works on a per-event basis though, not internally WITHIN events, so your two options as I see it are:

Use SEDCMD to remove the parts of the events that you don't want. This works very much like when using SEDCMD to anonymize data, so have a look at http://docs.splunk.com/Documentation/Splunk/latest/Data/Anonymizedatausingconfigurationfiles and modify to suit your needs
Instead of having Splunk read the file and remove unwanted stuff, setup a scripted input that does this before events arrive to Splunk.

Re: Indexing only specific fields in an event

pdash — Tue, 19 Feb 2013 22:52:19 GMT

yea but anonymizing wont send it to null queue right? I dont want them counting to my indexing volume

Re: Indexing only specific fields in an event

Ayn — Tue, 19 Feb 2013 22:55:00 GMT

The license counts against how much data is indexed. So, any way you remove data before being indexed results in less data indexed and so results in less data being counted against your license limit.