Getting Data In

How to exclude from monitoring empty files?

apakhomov
Path Finder

Hello,

Monitor folders have many empty files. These files may be filled in the future. So I can't add them to a blacklist.
As result the log file splunkd.log has huge amount messages:

INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'.

I don't want to reduce the log level.
Is it possible to exclude from monitoring empty files to reduce the message count in the log?

--
Best regards, Artem.

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

View solution in original post

0 Karma

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

0 Karma

apakhomov
Path Finder

Hello, sorry for the delay.

It is fantastic solution and I marked this as a solution. But unfortunately I can't to change the source files (even empty).

Best regards, Artem.

0 Karma

woodcock
Esteemed Legend

Whatever you do, DO NOT use ignoreOlderThan because once Splunk ignores a file via this control, it will never check it again.

0 Karma

apakhomov
Path Finder

ok, thank you for the useful information.

Best regards, Artem.

0 Karma

apakhomov
Path Finder

Universal forwarder start to reindex files when I commented the ignoreOlderThan parameter. I saw it today. The bitter experience with another task.

Best regards, Artem.

0 Karma
Get Updates on the Splunk Community!

Aligning Observability Costs with Business Value: Practical Strategies

 Join us for an engaging Tech Talk on Aligning Observability Costs with Business Value: Practical ...

Mastering Data Pipelines: Unlocking Value with Splunk

 In today's AI-driven world, organizations must balance the challenges of managing the explosion of data with ...

Splunk Up Your Game: Why It's Time to Embrace Python 3.9+ and OpenSSL 3.0

Did you know that for Splunk Enterprise 9.4, Python 3.9 is the default interpreter? This shift is not just a ...