How to reduce the daily ingestion on Splunk cloud?

[VinodTiwari]

Hi,

We use splunk cloud and our daily ingestion limit is 800 GB, we are ingesting about 100 GB over the limit. I'm looking for suggestions as in what can be done to reduce the ingestion. I did some digging and found we were sending debug logs which we didn't need, but that is just 10 GB per day. I'm fairly new to splunk and seeking help in understanding how this can be achieved.

Thanks

[richgalloway]

Check the inputs.conf files on your forwarders. Your monitor:// stanzas may be looking at more files than they need to. If you rotate your log files, make sure you are not ingesting the rolled logs ("foo.log.1", for example).

[skoelpin]

I've seen this happen at a customer site. They were monitoring everything with a * and when the logs rolled, the fishbucket got confused and thought the rolled log was a new file not seen before and massively increased volumes. The solution was to monitor the active log file and to use the crcSalt = <SOURCE> attribute to add more uniqueness to the hash

[skoelpin]

Hi @VinodTiwari, welcome to Answers!

How many servers and files are you currently monitoring? I would recommend leaving only the mission critical stuff logging first and turn everything else off. You should then find out what you don't need to monitor and exclude it.

You most likely have a * in your inputs.conf to monitor all the files, so you should remove that and write an expression to capture the files you want to ingest

You should also look into turning off debug mode if not needed and also use nullQueue to exclude certain text from logs that you don't need

https://docs.splunk.com/Documentation/SplunkCloud/6.6.3/Data/Getstartedwithgettingdatain

https://answers.splunk.com/answers/305311/how-to-send-events-to-the-nullqueue-on-indexer.html