topic Re: How to drop all entries to a specific index? in Getting Data In

How to drop all entries to a specific index?

mlindsey — Thu, 29 May 2014 17:27:45 GMT

We've reached our license limit. So, at the indexer, I want to drop all log entries destined to a specific index. Documentation is clear how to do that on a heavy forwarder, for example, but I haven't found any documentation of how to drop all traffic to a specific index at the indexer. Props.conf looked promising but it doesn't support an index key. In props.conf, I was expecting that I could create a stanza like this:

[index::development] # This key is not listed in the props.conf.spec

TRANSFORMS-blackhole = blackhole

and in transforms.conf:

[blackhole]
REGEX = .
DEST_KEY = queue
FORMAT = nullQueue

It just seems there has to be a way, but I haven't been able to discover it.

Re: How to drop all entries to a specific index?

jhupka — Thu, 29 May 2014 17:37:27 GMT

Wouldn't your REGEX=. just match single character events for sending to nullQueue? Maybe try REGEX=.* to make it more greedy and match everything.

Re: How to drop all entries to a specific index?

mlindsey — Thu, 29 May 2014 18:33:05 GMT

The REGEX is really irrelevant in this case because the example is a fantasy not an actual configuration. I just hoped it would be that way. The real issue is routing all traffic destined to one index to the nullQueue.

Re: How to drop all entries to a specific index?

somesoni2 — Thu, 29 May 2014 19:19:49 GMT

As far as I know, the filtering can be done at source, sourcetype or host level, its not possible for index level. I would be interested to see if there really is a way.

Re: How to drop all entries to a specific index?

jhupka — Fri, 30 May 2014 14:57:36 GMT

So I don't really have a quick and dirty answer to shutting off an index that doesn't come with its own caveats. E.g. if I had to do this I might try something like:

Just manually off the inputs going to that index and deploy it out real quick using DS.
If you don't care about anything in that index for now, you could just remove its config from your index.conf. But you'll get warnings when stuff goes to that index, but those events get dropped and I'm pretty sure don't affect license. Just don't delete the index's directories so when you re-enable the index old data is still there

Re: How to drop all entries to a specific index?

mlindsey — Mon, 28 Sep 2020 16:48:54 GMT

The following worked just great! Kudos to James Brodsky and Splunk Support.

In props.conf:

# This stanza drops alls events destined for the 'development' index.

#

[host::*]

TRANSFORMS-dropindex = senddevidxtonull

In transforms.conf:

[senddevidxtonull]

SOURCE_KEY=_MetaData:Index

REGEX=development

DEST_KEY=queue

FORMAT=nullQueue