Monitoring files during specific hours

[peter_gianusso]

Is it possible to only forward certain files during a specific time period?

For instance, I only want the forwarder to monitor file XYZ between 9 AM to 5 PM.

I don't want to approach this from a search standpoint. I want the forwarder to basically exclude the file from 5 PM to 9 AM. Not touch it at all.

Thanks!!

[Drainy]

At a high level, yes you can.

Essentially you would need to create a script that ran on a schedule to hit the REST endpoint to disable or activate an input for the files in question. For those inputs you need to add the line;
followTail = true

This will make Splunk only read from where the file currently is and not consume any older data, once disabled it will stop reading the data in.
Alternatively you could have a script running that reads the original log and writes it to a second location, this could then be easily configured to only output to the second file during the times you specify.

[mloven_splunk]

I was going to suggest using followTail, but the docs state that:
"* DO NOT leave followTail enabled in an ongoing fashion."

I wasn't sure how much of a negative effect it would have to leave it enabled for 8 hours a day.

Thoughts?

[lmyrefelt]

cron job to activate / deactivate the input ? 🙂

[mloven_splunk]

There's nothing built into Splunk to accommodate this as far as I know.

You could hack something together that would enable/disable an input via a script, but as soon as you re-enable the input, it would just index all of the data from the time you don't care about.