File age/processing measurement

[tyronetv]

Have a set of directories that act as "holding" or "pending" directories for file transfer. Essentially we transfer the file and then put a copy of it in the /pending/ directory awaiting the remote site to process and confirm. This process and confirm can take between 10 minutes and 2 hours. Once the files are processed and confirmed we remove the file from /pending and move it to /sent.

What I want to do is to monitor the /pending directory. Capture the file's initial receipt and track until it is removed. I don't need to index the file, CRC the file, or any of that, I just need to say, "Hey! File xxxx.zip is here" and "Hey! File xxxx.zip is no longer here." so I can pull some metrics on how long the process takes as well as set up alerts for when it takes to long.

Anyone done anything like this and have any suggestions?

Thanks!

[lguinn2 · Accepted solution]

Use the fschange monitoring - it should work great, because this is exactly what it is designed to do! You can read up on it n the manuals, but you probably don't need any of the advanced options. Just do this

[fschange:/xxx/yyy/pending]
pollPeriod=60
sourcetype=PendingFileMonitor

This will create an event every time a file is added, deleted or changed in the directory. The events are nicely formatted and have a field identifying the exact file name and what the change was. pollPeriod is how often Splunk should check the directory for changes (in seconds). Where I have specified /xxx/yyy/pending, you should put the absolute path to the directory.

[lguinn2]

Good point about the phase-out, but my guess is that it will be at least a year. That's an absolute SWAG, based on how often major updates have occurred in the past. I am hoping (again, no data here) that there will be a good replacement for this functionality by then. See http://splunk-base.splunk.com/answers/63874/why-is-fschange-a-deprecated-feature-in-splunk-50 for more info

[tyronetv]

With fschange being phased out (via my Splunk> 5.0 notes) I wonder how much longer this will be valid.

[bmacias84]

Keep in mind you cannot simultaneously watch a directory using both fschange monitor and monitor. Also depending on how many files are in your directory and sub-dirs(if recursive is enabled) CPU of your host system could be impacted. In the event that occurs look at adding the following settings to fschange stanza; filesPerDelay and delayInMills .