Getting Data In

Monitoring files during specific hours

peter_gianusso
Communicator

Is it possible to only forward certain files during a specific time period?

For instance, I only want the forwarder to monitor file XYZ between 9 AM to 5 PM.

I don't want to approach this from a search standpoint. I want the forwarder to basically exclude the file from 5 PM to 9 AM. Not touch it at all.

Thanks!!

0 Karma

Drainy
Champion

At a high level, yes you can.

Essentially you would need to create a script that ran on a schedule to hit the REST endpoint to disable or activate an input for the files in question. For those inputs you need to add the line;
followTail = true

This will make Splunk only read from where the file currently is and not consume any older data, once disabled it will stop reading the data in.
Alternatively you could have a script running that reads the original log and writes it to a second location, this could then be easily configured to only output to the second file during the times you specify.

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

I was going to suggest using followTail, but the docs state that:
"* DO NOT leave followTail enabled in an ongoing fashion."

I wasn't sure how much of a negative effect it would have to leave it enabled for 8 hours a day.

Thoughts?

0 Karma

lmyrefelt
Builder

cron job to activate / deactivate the input ? 🙂

0 Karma

mloven_splunk
Splunk Employee
Splunk Employee

There's nothing built into Splunk to accommodate this as far as I know.

You could hack something together that would enable/disable an input via a script, but as soon as you re-enable the input, it would just index all of the data from the time you don't care about.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!