Getting Data In

How can I troubleshoot why suddenly 8 of 10 subfolders with proxy logs have stopped being indexed?

daniel_augustyn
Contributor

I've been sending proxy logs to the FTP server and from there I installed an universal forwarder to send the logs to the Splunk indexers. They are all in a gz format. Everything was working fine until a day when I've noticed that proxy logs stopped getting indexed. There are about 10 subfolders and only 2 of them are still getting indexed, and the rest of the proxy logs had stopped getting indexed on the same day. How should I troubleshoot this?

Not sure why some of the subfolders with gz files (proxy logs from each site) has stopped getting indexed and the rest is still going.

0 Karma
1 Solution

daniel_augustyn
Contributor

The issue was because I added to much to my stanza for monitoring too many files at once and Splunk basically filled up the buffer. Since proxy logs are in gz format, it took long time for Splunk to catch up. Splunk will need to finish a single gz file before it could move to the next one. I also increased thruput in limints.conf.

View solution in original post

0 Karma

daniel_augustyn
Contributor

The issue was because I added to much to my stanza for monitoring too many files at once and Splunk basically filled up the buffer. Since proxy logs are in gz format, it took long time for Splunk to catch up. Splunk will need to finish a single gz file before it could move to the next one. I also increased thruput in limints.conf.

0 Karma
Get Updates on the Splunk Community!

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...