Getting Data In

How to exclude from monitoring empty files?

apakhomov
Path Finder

Hello,

Monitor folders have many empty files. These files may be filled in the future. So I can't add them to a blacklist.
As result the log file splunkd.log has huge amount messages:

INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'.

I don't want to reduce the log level.
Is it possible to exclude from monitoring empty files to reduce the message count in the log?

--
Best regards, Artem.

Tags (1)
0 Karma
1 Solution

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

View solution in original post

0 Karma

woodcock
Esteemed Legend

You could dump some filler into each file:

echo "FILLER: This is not real data but just filler text to suppress this log: INFO  WatchedFile - File too small to check seekcrc, probably truncated. Will re-read entire file='<filename>'." > <filename>.

Then configre props.conf and transforms.conf to send these events to nullQueue.

0 Karma

apakhomov
Path Finder

Hello, sorry for the delay.

It is fantastic solution and I marked this as a solution. But unfortunately I can't to change the source files (even empty).

Best regards, Artem.

0 Karma

woodcock
Esteemed Legend

Whatever you do, DO NOT use ignoreOlderThan because once Splunk ignores a file via this control, it will never check it again.

0 Karma

apakhomov
Path Finder

ok, thank you for the useful information.

Best regards, Artem.

0 Karma

apakhomov
Path Finder

Universal forwarder start to reindex files when I commented the ignoreOlderThan parameter. I saw it today. The bitter experience with another task.

Best regards, Artem.

0 Karma
Get Updates on the Splunk Community!

Tech Talk Recap | Mastering Threat Hunting

Mastering Threat HuntingDive into the world of threat hunting, exploring the key differences between ...

Observability for AI Applications: Troubleshooting Latency

If you’re working with proprietary company data, you’re probably going to have a locally hosted LLM or many ...

Splunk AI Assistant for SPL vs. ChatGPT: Which One is Better?

In the age of AI, every tool promises to make our lives easier. From summarizing content to writing code, ...