topic Re: Monitoring files during specific hours in Getting Data In

Monitoring files during specific hours

peter_gianusso — Thu, 01 Aug 2013 16:29:16 GMT

Is it possible to only forward certain files during a specific time period?

For instance, I only want the forwarder to monitor file XYZ between 9 AM to 5 PM.

I don't want to approach this from a search standpoint. I want the forwarder to basically exclude the file from 5 PM to 9 AM. Not touch it at all.

Thanks!!

Re: Monitoring files during specific hours

mloven_splunk — Thu, 01 Aug 2013 17:09:34 GMT

There's nothing built into Splunk to accommodate this as far as I know.

You could hack something together that would enable/disable an input via a script, but as soon as you re-enable the input, it would just index all of the data from the time you don't care about.

Re: Monitoring files during specific hours

lmyrefelt — Fri, 02 Aug 2013 07:02:15 GMT

cron job to activate / deactivate the input ? 🙂

Re: Monitoring files during specific hours

Drainy — Fri, 02 Aug 2013 08:22:43 GMT

At a high level, yes you can.

Essentially you would need to create a script that ran on a schedule to hit the REST endpoint to disable or activate an input for the files in question. For those inputs you need to add the line;
followTail = true

This will make Splunk only read from where the file currently is and not consume any older data, once disabled it will stop reading the data in.
Alternatively you could have a script running that reads the original log and writes it to a second location, this could then be easily configured to only output to the second file during the times you specify.

Re: Monitoring files during specific hours

mloven_splunk — Fri, 02 Aug 2013 13:37:59 GMT

I was going to suggest using followTail, but the docs state that:
"* DO NOT leave followTail enabled in an ongoing fashion."

I wasn't sure how much of a negative effect it would have to leave it enabled for 8 hours a day.

Thoughts?