Getting Data In

How do I know if universal forwarder has finished reading and complteted forwarding data to indexers?

melonman
Motivator

Hi,

I have been looking for information about REST API point to know if UF has completed reading the file and sending events to indexer, or simply waiting for events to be logged to monitored file:

/services/admin/inputstatus/TailingProcessor:FileStatus 

I am trying to track if universal forwarder is reading files monitored or completed sending events.

some key shows type="open file", and others showing "finished reading".
Could anyone know the difference between "open file" and "finished reading"?
If there are more status for file input, I would like to know the information about the status of file being monitored.

<s:key name="/root/test.log">
<s:dict>
<s:key name="file position">2252994</s:key>
<s:key name="file size">2252994</s:key>
<s:key name="percent">100.00</s:key>
<s:key name="type">**open file**</s:key>
</s:dict>
</s:key>

and,

<s:key name="/opt/splunkforwarder/var/log/splunk/license_audit.log">
<s:dict>
<s:key name="file position">0</s:key>
<s:key name="file size">0</s:key>
<s:key name="parent">$SPLUNK_HOME/var/log/splunk/splunkd.log</s:key>
<s:key name="percent">100</s:key>
<s:key name="type">**finished reading**</s:key>
</s:dict>
</s:key>

Any information would be appreciated.

Tags (1)
0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

Splunk assumes that a new event can always be added to any file, so it will keep monitoring every files.

about the REST endpoint, when the percent is 100% :

  • "finished reading" means that the file has been read and forwarded till the end
  • "open file" means the same, but in addition the handle on the file is still open (because it has been less than 3 seconds, or because an option in inputs.conf request to keep the file open to tail new events, or because the file has just being reopen to check if they were any update or rotation)

View solution in original post

yannK
Splunk Employee
Splunk Employee

Splunk assumes that a new event can always be added to any file, so it will keep monitoring every files.

about the REST endpoint, when the percent is 100% :

  • "finished reading" means that the file has been read and forwarded till the end
  • "open file" means the same, but in addition the handle on the file is still open (because it has been less than 3 seconds, or because an option in inputs.conf request to keep the file open to tail new events, or because the file has just being reopen to check if they were any update or rotation)

melonman
Motivator

Thank you for your answer, I am looking for this type of information in a documentation, but I still can not find it. I would really appreciate if you point me to the location in a doc or any other location.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Get Agentic with Splunk Lantern: Connect to Cisco Cloud Control, Transform ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

July Community Events: Master ITSI 5.0 & Automate Splunk

Struggling with alert fatigue or feeling like you're spending more time on infrastructure maintenance than ...

New Release of Federated Search: Bringing Splunk Analytics to More of Your Data

Organizations today are generating more data than ever and storing it across cloud object stores, data lakes, ...