topic Re: How to exclude from monitoring empty files? in Getting Data In

How to exclude from monitoring empty files?

apakhomov — Thu, 02 Jul 2015 08:58:05 GMT

Hello,

Monitor folders have many empty files. These files may be filled in the future. So I can't add them to a blacklist.
As result the log file splunkd.log has huge amount messages:

INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'.

I don't want to reduce the log level.
Is it possible to exclude from monitoring empty files to reduce the message count in the log?

--
Best regards, Artem.

Re: How to exclude from monitoring empty files?

woodcock — Thu, 02 Jul 2015 12:51:58 GMT

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

Re: How to exclude from monitoring empty files?

apakhomov — Wed, 15 Jul 2015 07:15:18 GMT

Hello, sorry for the delay.

It is fantastic solution and I marked this as a solution. But unfortunately I can't to change the source files (even empty).

Best regards, Artem.

Re: How to exclude from monitoring empty files?

woodcock — Wed, 15 Jul 2015 14:41:39 GMT

Whatever you do, DO NOT use ignoreOlderThan because once Splunk ignores a file via this control, it will never check it again.

Re: How to exclude from monitoring empty files?

apakhomov — Wed, 15 Jul 2015 14:48:34 GMT

ok, thank you for the useful information.

Best regards, Artem.

Re: How to exclude from monitoring empty files?

apakhomov — Thu, 16 Jul 2015 05:55:13 GMT

Universal forwarder start to reindex files when I commented the ignoreOlderThan parameter. I saw it today. The bitter experience with another task.

Best regards, Artem.