I've set up a heavy forwarder on "stage2" (linux machine) and a central Splunk instance on "stage1" (another linux machine). I'm trying to monitor the following folder on stage2:
/x/web/stage2/logs/
and I want to pick up the following files:
ALERT.log.15-08-01.log
ALERT.log.15-08-06.log
ALERT.log.15-08-07.log
ALERT.log.15-08-08.log
ALERT.log.15-08-09.log
ALERT.log.15-08-10.log
ALERT.log.15-08-11.log
ALERT.log.15-08-12.log
ALERT.log.15-08-13.log
ALERT.log.15-08-14.log
ALERT.log.15-08-15.log
ALERT.log.15-08-16.log
ALERT.log.15-08-17.log
ALERT.log.15-08-18.log
ALERT.log.15-08-19.log
ALERT.log.15-08-20.log
ALERT.log.15-08-21.log
ALERT.log.15-08-22.log
ALERT.log.15-08-23.log
ALERT.log.15-08-24.log
ALERT.log
It is important that the rollover/historical logs get picked up. (FYI I had no say in the naming convention used for them)
Steps I took:
On stage2, under Settings -> Data Inputs -> Files & Directories, the monitoring rule shows a high enough number to where I believe stage2 is monitoring all the files I want it to, but only the one ALERT.log file makes it to stage1.
Is there something that prevents Splunk from picking up logs named a certain way? Anyone with any ideas about what might be going on here?
First check that the splunk user has permission to read all of these log files.
Here is what I would put in the inputs.conf file because all of your logs end in .log.
[monitor:///x/web/stage2/logs/]
disabled = false
blacklist = .(gz|bz2|swp|z|zip)$
whitelist = .log$