How to forward all events to a single index, but filter out all splunkd sourcetype events that contain "INFO"?

[mwilson788]

We are currently using props.conf and transforms.conf to combine all non-internal ingest into a single index on our heavy forwarder (coming from 100s of universal forwarders).

We are doing so with the following config:

props.conf
[default]
TRANSFORMS-index = setdefaultindex

transforms.conf
[setdefaultindex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = customindexname

All was working well until we decided to start filtering out all splunkd sourcetype events that contain INFO. It seems they are forwarded on regardless of the filter we put in place due to the config above. Is there a way around this?

[jayannah · Accepted solution]

I shall described 2 solutions for your problem-1. However, I do prefer option-1 below.

Option-1: If you don't specify index name in the inputs.conf, then all the events goto the default index. By default, the default index is 'main', but you can configure "customindex" as default index so that all events which arrives without index name in the metadata will go into the "customindex" or default index whatever you configure. Use below below props & transforms just to filter out (drop) INFO events.

props.conf
[default]
TRANSFORMS-index = drop_info_events, send_all_events

transforms.conf
[drop_info_events]
REGEX = INFO
DEST_KEY = queue
FORMAT = nullQueue

[send_all_events]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

option-2: Use below configure where the customer index name is mentioned in the configuration files.

props.conf
[default]
TRANSFORMS-index = setdefaultindex, drop_info_events, send_all_events

transforms.conf
[setdefaultindex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = customindexname

[drop_info_events]
REGEX = INFO
DEST_KEY = queue
FORMAT = nullQueue

[send_all_events]
REGEX = .
DEST_KEY = queue
FORMAT = indexQueue

[mwilson788]

I should say Option 2 is the one I tried. I'm afraid I'm unable to change the inputs.conf file on the UFs as mentioned in Option 1 (I agree this would be ideal).

[mwilson788]

I really thought this might work but alas, the events still poor in. It's like the filter is being ignored completely...

[jayannah]

Did you restart your Heavy forwarder after putting these changes?

[mwilson788]

Of course. Still tinkering. I appreciate the effort. If you think of anything at all, it would be appreciated.

[jayannah]

Did you remove the double quotes in the REGEX = "INFO" line in props.conf??

[mwilson788]

I did actually. I just got this to work with a bit of tweaking.

Answer here for others who may have this issue:

props.conf
[default]
TRANSFORMS-index = setdefaultindex, setnull

transforms.conf
[setdefaultindex]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = indexname

[setnull]
REGEX = INFO
DEST_KEY = queue
FORMAT = nullQueue

This was very close to your solution so I give you all the credit jayannah! Thanks so much!

[MuS]

Hi, since you only provided the example for the default index, this is hard to solve.
Take a look at the docs about routing and filtering, which contains an example on how to filter events and send them into the darkness of digital universe or the null queue http://docs.splunk.com/Documentation/Splunk/6.2.1/Forwarding/Routeandfilterdatad#Filter_event_data_a...

cheers, MuS

[mwilson788]

Thank you for your response. Actually, the document you reference is what I've been using to the letter.

The following was added to props.conf for filtering:
[splunkd]
TRANSFORMS-null = setnull

And to transforms.conf:
[setnull]
REGEX = \"INFO\"
DEST_KEY = queue
FORMAT = nullQueue

According to the doc, this should work but, does nothing. Am I overlooking something simple here?

As a side note, I even went so far as to set priority on the stanzas as a test. No luck there either.

[jayannah]

REGEX = "INFO" is exactly match "INFO" with quotes. So, if your event doesn't have quotes around INFO, then use REGEX = INFO

Also, you have user another transforms to forwarder the remaining event. Please the configurations I have explained in the answer.

let me know it works.