Getting Data In

How to exclude a sourcetype from being indexed?

skoelpin
SplunkTrust
SplunkTrust

I have a forwarder on 3 different servers which grabs all the data coming from those servers. There is 1 specific sourcetype and source both called 'WinEventLog:Application' which we do not want indexed.

Where in the forwarder do I exclude this source/sourcetype from being indexed?

0 Karma
1 Solution

woodcock
Esteemed Legend

The answer by @somesoni2 should work and is the proper way to do it because it stops the forwarders from doing work that is not necessary which this alternate solution does not do. The solution below will have the forwarders send the data in and then the indexers will trash all events for the undesirable sourcetype:

In props.conf do this:

[WinEventLog:Application]
TRANSFORMS-remove_events = remove_events 

In transforms.conf do this:

[remove_events]
REGEX = *
DEST_KEY = queue
FORMAT = nullQueue

View solution in original post

woodcock
Esteemed Legend

The answer by @somesoni2 should work and is the proper way to do it because it stops the forwarders from doing work that is not necessary which this alternate solution does not do. The solution below will have the forwarders send the data in and then the indexers will trash all events for the undesirable sourcetype:

In props.conf do this:

[WinEventLog:Application]
TRANSFORMS-remove_events = remove_events 

In transforms.conf do this:

[remove_events]
REGEX = *
DEST_KEY = queue
FORMAT = nullQueue

jawaharas
Motivator

Thank you.

However below configuration line errored out.
[remove_events]
REGEX = *

By modifying as below, it works.
[remove_events]
REGEX = (?s).*

0 Karma

skoelpin
SplunkTrust
SplunkTrust

I just looked at the 3 hosts which have the WinEventLog:Application and these are being indexed in index=main while my legitimate events are being indexed in the index I specified in the inputs.conf file.. So maybe this would explain why I can't find it.. But why is data being indexed if I didn't specify the source or sourcetype in the inputs.conf?

0 Karma

woodcock
Esteemed Legend

Depending on your OS and the package that you download, Splunk is preconfigured to send some things in by default. Most of the time, this is what people desire so it makes sense.

0 Karma

skoelpin
SplunkTrust
SplunkTrust

Should I do this on the Indexer or forwarder?

0 Karma

woodcock
Esteemed Legend

On the indexers and then restart Splunk services.

0 Karma

somesoni2
Revered Legend

Find the inputs.conf file on the forwarder which defines the data input (what data is monitored and where its stored). It could be in Splunk_Home/etc/system/local OR Splunk_Home/etc/apps/SomeAppName/local OR / Splunk_Home/etc/apps/SomeAppName/default. Once found look for the sourcetype name and disable/remove the data input which is using that sourcetype. Restart the forwarder once done.

skoelpin
SplunkTrust
SplunkTrust

Yeah I have that pulled open and the sourcetype is already defined as sourcetype=ATG_Message_Log.. I see nothing on any of the boxes which show the source or sourcetype as WinEventLog:Application

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...