Getting Data In

What happens if I restart the universal forwarder while it is processing a file?

lyndac
Contributor

Here's my setup: 1 search head, 4 indexers, 1 universal forwarder

The UF is trying to index a large file (2G), I'm seeing the "Current data throughput (256kb/s) has reached maxKBps. AS a result data forwarding may be throttled."

If I increase the maxKBps in limits.conf, and restart the UF, will I lose the data that was waiting to be forwarded? Will the Splunk UF pick up in the same spot of the file to keep trying?

My inputs.conf is

[batch:///opt2/ingest/json-data]
    index=foo
    sourcetype=json-foo
    move_policy=sinkhole

and my outputs.conf is:

[tcpout]
defaultGroup=primary_indexers
useACK=true

[tcpout:primary_indexers]
autoLBFrequency=30
server=server1:9997,server2:9997,server3:9997,server4:9997

(I actually have the server ips in the server list above).

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Splunk can resume from the position it was on the monitored file.
however with the sinkhole, it should delete the file once compelted.

Do you see the file deleted yet ?
to be safe, I would recommend to make a copy of the file
When you stop splunk, do not kill the process let it empty it's output queue to ensure that no events are missed.
Then if needed, reindex the file later.

View solution in original post

0 Karma

yannK
Splunk Employee
Splunk Employee

Splunk can resume from the position it was on the monitored file.
however with the sinkhole, it should delete the file once compelted.

Do you see the file deleted yet ?
to be safe, I would recommend to make a copy of the file
When you stop splunk, do not kill the process let it empty it's output queue to ensure that no events are missed.
Then if needed, reindex the file later.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Index This | What travels the world but is also stuck in place?

April 2026 Edition  Hayyy Splunk Education Enthusiasts and the Eternally Curious!   We’re back with this ...

Discover New Use Cases: Unlock Greater Value from Your Existing Splunk Data

Realizing the full potential of your Splunk investment requires more than just understanding current usage; it ...

Continue Your Journey: Join Session 2 of the Data Management and Federation Bootcamp ...

As data volumes continue to grow and environments become more distributed, managing and optimizing data ...