All Apps and Add-ons

App to stop unwanted logs ingestion ?

AL3Z
Builder

Hi,

Is there any app in Splunk base to analyze the logs in my Splunk ES to stop the unwanted logs ingestion ?

Thanks

Labels (1)
0 Karma

inventsekar
Ultra Champion

the "unwanted logs" is a very vague term and the ES app definitely got no app to monitor this vagueness. 

as said by above reply, you should fine-tune what to ingest and what not to ingest(and send it to null-queue). 

AL3Z
Builder

As we are on the Splunk cloud GCP its won't supports the  null-queue.

0 Karma

isoutamo
SplunkTrust
SplunkTrust

Hi

all splunk instances should accept nullQeue. But you must define this parameter/transformation on the first full splunk instance from UF/data source to indexers. It could be HF or indexer.  
r. Ismo

richgalloway
SplunkTrust
SplunkTrust

Where did you learn this?  AFAIK, Splunk Enterprise and Splunk Cloud on all platforms supports null queue.

---
If this reply helps you, Karma would be appreciated.

inventsekar
Ultra Champion

Please select the logs you wanted to ingest... fine-tune it. thats all. its very simple actually. 

or

simply, go with filesize. 

order all your ingested files by their size(in DMC----license consumed by log sources in last 24hrs dashboard)... if you see any logs in top 10, top20 which is an unwanted log file, then... remove it from inputs.conf (you may be using wildcards in inputs.conf.... if so, add a blacklist for that file)

0 Karma

chris_barrett
Communicator

Short answer - no, there isn't.

Longer answer - but you could write an app to do it.

 

Assuming that you don't care for events of the sourcetype foo, you could set up an app on your indexers that looks like:

props.conf

[foo]
TRANSFORM-route_to_nullQueue = route_to_nullQueue

transforms.conf

[route_to_nullQueue]
INGEST_EVAL = queue := "nullQueue"

 

Once active, events of sourcetype foo won't be ingested and won't count towards your daily ingest license.

Get Updates on the Splunk Community!

Observability | How to Think About Instrumentation Overhead (White Paper)

Novice observability practitioners are often overly obsessed with performance. They might approach ...

Cloud Platform | Get Resiliency in the Cloud Event (Register Now!)

IDC Report: Enterprises Gain Higher Efficiency and Resiliency With Migration to Cloud  Today many enterprises ...

The Great Resilience Quest: 10th Leaderboard Update

The tenth leaderboard update (11.23-12.05) for The Great Resilience Quest is out >> As our brave ...