Getting Data In

How to exclude from monitoring empty files?

apakhomov
Path Finder

Hello,

Monitor folders have many empty files. These files may be filled in the future. So I can't add them to a blacklist.
As result the log file splunkd.log has huge amount messages:

INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'.

I don't want to reduce the log level.
Is it possible to exclude from monitoring empty files to reduce the message count in the log?

--
Best regards, Artem.

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

View solution in original post

0 Karma

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

0 Karma

apakhomov
Path Finder

Hello, sorry for the delay.

It is fantastic solution and I marked this as a solution. But unfortunately I can't to change the source files (even empty).

Best regards, Artem.

0 Karma

woodcock
Esteemed Legend

Whatever you do, DO NOT use ignoreOlderThan because once Splunk ignores a file via this control, it will never check it again.

0 Karma

apakhomov
Path Finder

ok, thank you for the useful information.

Best regards, Artem.

0 Karma

apakhomov
Path Finder

Universal forwarder start to reindex files when I commented the ignoreOlderThan parameter. I saw it today. The bitter experience with another task.

Best regards, Artem.

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...