Getting Data In

How to set time interval on a universal forwarder to check a specific file in directory?

horsefez
Motivator

Hi,

I have one application at my company which logs only once a day.
It hereby overwrites the file of the day before.
How can I tell the universal forwarder to grab a specific file only once a day?
I want to set an interval, there is no need for an exact point in time.

Thank you!

Best Regards,
pyro_wood

0 Karma
1 Solution

lguinn2
Legend

If you set the universal forwarder to monitor the file, it will check it throughout the day. When the file changes completely, Splunk will index the entire new file at some point after the change.

Note that Splunk checks the first 256 bytes of the file to check to see whether the file has been replaced or just appended. So if the first part of the file is always the same, Splunk may not realize that it really is a new file. You can fix this by setting the following in the inputs.conf stanza that is monitoring the file:

initCrcLength = 1024

Although you may need to set it to something larger - it needs to be a number of bytes that will force Splunk to look beyond any common header.

There are other settings that can force Splunk to always re-index the entire file when it changes (eg., crcSalt). You can find out more about this by reading about inputs.conf in the Admin manual.

Although you can set up Splunk "to check at an interval" by using scripts, but that is kludgy compared to just setting a monitor input. As @somesoni2 suggests, this is the best practice. The monitor input is reliable and low overhead.

View solution in original post

0 Karma

lguinn2
Legend

If you set the universal forwarder to monitor the file, it will check it throughout the day. When the file changes completely, Splunk will index the entire new file at some point after the change.

Note that Splunk checks the first 256 bytes of the file to check to see whether the file has been replaced or just appended. So if the first part of the file is always the same, Splunk may not realize that it really is a new file. You can fix this by setting the following in the inputs.conf stanza that is monitoring the file:

initCrcLength = 1024

Although you may need to set it to something larger - it needs to be a number of bytes that will force Splunk to look beyond any common header.

There are other settings that can force Splunk to always re-index the entire file when it changes (eg., crcSalt). You can find out more about this by reading about inputs.conf in the Admin manual.

Although you can set up Splunk "to check at an interval" by using scripts, but that is kludgy compared to just setting a monitor input. As @somesoni2 suggests, this is the best practice. The monitor input is reliable and low overhead.

0 Karma

horsefez
Motivator

Thank you for this helpful reply 🙂

0 Karma

somesoni2
Revered Legend

Do you see any issue with regular options of monitoring OR batch?

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Design, Compete, Win: Submit Your Best Splunk Dashboards for a .conf26 Pass

Hello Splunkers,  We’re excited to kick off a Splunk Dashboard contest! We know that dashboards are a primary ...

May 2026 Splunk Expert Sessions: Security & Observability

Level Up Your Operations: May 2026 Splunk Expert Sessions Whether you are refining your security posture or ...

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...