Getting Data In

Is it possible that the Splunk forwarder caches old events and resends the data again and again?

sanjay_shrestha
Contributor

We have a batch script which monitors files under some folder and then creates a log file with the file name and file creation time information every 5 mins. Currently, that batch script is creating an empty log file as there are no files in the inspected folder, but there are events coming from that log file to Splunk every 5 minutes.

Is it possible that the Splunk forwarder caches old events and re-sends again and again?

0 Karma

fdi01
Motivator

After sending a data block, the forwarder maintains a copy of the data in its wait queue until it receives an acknowledgment. In the meantime, it continues to send additional blocks as usual. If the forwarder doesn't get acknowledgment for a block within 300 seconds (by default), it closes the connection. You can change the wait time by setting the readTimeout attribute in outputs.conf.

0 Karma

martin_mueller
SplunkTrust
SplunkTrust

That sounds unlikely as long as indexers are working fine. What's the source field of the repeated data? Can you post the input configuration for that source?

If your indexers are not working fine and you use useAck then there is a small chance of duplicates: http://docs.splunk.com/Documentation/Splunk/6.2.2/Forwarding/Protectagainstlossofin-flightdata#The_p...
...not regularly every five minutes though, and this would be logged in the forwarder's splunkd.log.

0 Karma

sanjay_shrestha
Contributor

It started at noon and Splunk was getting until 12:55 PM however it stopped after 1 PM. Does this mean anything?

0 Karma
Get Updates on the Splunk Community!

Adoption of RUM and APM at Splunk

    Unleash the power of Splunk Observability   Watch Now In this can't miss Tech Talk! The Splunk Growth ...

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...