I must be doing something wrong. Splunk is seeing and indexing the first log file it finds and nothing else after within the same folder and nothing else after in the other folders. Any advice? Thank you. -Sean
Full path to log files are (sact01 - sact15):
/logs/sam/ct/sact01/section01/web /logs/sam/ct/sact01/sec-02/web /logs/sam/ct/sact01/sec-03/web /logs/sam/ct/sact02/section01/web /logs/sam/ct/sact02/sec-02/web /logs/sam/ct/sact02/sec-03/web
Log files name format within /web (4 months worth, but i only need 7 days):
request.log.2017-02-28-13-16-04 request.log.2017-03-01-08-12-04 request.log.2017-03-02-13-33-04
[monitor:///logs/sam/ct/sact0*/.../web] _TCP_ROUTING = WestCoast0102 disabled = false index = test-i007 sourcetype = sam whitelist = request\.logs*$ crcSalt = <SOURCE> ignoreOlderThan = 7d
TailingProcessor - Adding watch on path: /logs/sam/ct. TailingProcessor - Parsing configuration stanza: monitor:///logs/sam/ct/sactws0*/.../web.
-- Log files name format within /web (4 months worth, but i only need 7 days):
That goes to best practices of the log directories and archiving. From Splunk perspective, it would be much better if this directory structure contains *only * the 7 days of data. The rest of the files should be moved to an archive area. If possible.
Sifting through a large directory structure and identifying the files to monitor is expensive and the software doesn't give you enough indications where it stands when it processes this directory, so we are in the dark.
You have told it to search for infinite depth and if you have many subdirectories, Splunk can get WAY too busy. Change it to this: