Getting Data In

Monitor size and last modification only for a single file on Unix filesystem

mflamerich
Explorer

We have some log files that we monitor as heartbeat for some daemon processes.

These files contain a large level of verbosity and tend to be quite large.

It turns out that we do not need to index the content, but we need to monitor that the files keep changing, either because of content or log rotation and creation of a new empty file with the same name.

We put few monitor:// lines in our inputs.conf and all has been fine for some time, but the processes that write these logs sometimes produce excessive debug information that is not required be indexed at all. These bursts of data have tripped our daily volume quota twice in a quarter, leaving us with no indexing for almost half a day.

We asked our users and even searched our _internal splunk logs and nobody has ever searched any content on those files. On the other hand, the alert when there has been no events on the monitors for some time has been proved very useful.

The reason I am posting this question is whether anyone knows of a way to monitor for a file size change and/or last modification time, that does not index anything of the file contents, but only the size of the file. We still need to run a search with an alert if the size of the file has not changed during a time interval.

Thanks.

Tags (2)
0 Karma

kristian_kolb
Ultra Champion

There is a deprecated function called fschange that can be used for this purpose. So, while I think it still works, support for this may be dropped in the next release.

http://docs.splunk.com/Documentation/Splunk/5.0.2/Data/Monitorchangestoyourfilesystem

Other than that, you should probably look at the file system auditing tools that come with your OS.

Hope this helps,

K

Ayn
Legend

Splunk hasn't dropped fschange. It's deprecated, which does NOT mean dropped. Last thing I heard on this is that Splunk will make sure fschange stays until it has its functionality replaced. So one way or another this functionality will continue to exist in the product.

I really think Splunk should be more clear about this, instead of having people resort to hearsay.

mflamerich
Explorer

Thanks for your suggestion. I think one of the reasons for posting my question is that Splunk deprecated fschange. I was wondering what was a reason for dropping fschange and if there was a better way to capture these kind of changes.

Given the lack of fschange alternatives, I think I'll create a scripted input.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...