How to correctly configure Splunk to monitor a directory with wildcards?

[rewritex]

I must be doing something wrong. Splunk is seeing and indexing the first log file it finds and nothing else after within the same folder and nothing else after in the other folders. Any advice? Thank you. -Sean

Full path to log files are (sact01 - sact15):

/logs/sam/ct/sact01/section01/web
/logs/sam/ct/sact01/sec-02/web
/logs/sam/ct/sact01/sec-03/web
/logs/sam/ct/sact02/section01/web
/logs/sam/ct/sact02/sec-02/web
/logs/sam/ct/sact02/sec-03/web

Log files name format within /web (4 months worth, but i only need 7 days):

request.log.2017-02-28-13-16-04
request.log.2017-03-01-08-12-04
request.log.2017-03-02-13-33-04

Inputs.conf

[monitor:///logs/sam/ct/sact0*/.../web]
_TCP_ROUTING = WestCoast0102
disabled = false
index = test-i007
sourcetype = sam
whitelist = request\.logs*$
crcSalt = <SOURCE>
ignoreOlderThan = 7d

index=_internal source="/opt/splunkforwarder/var/log/splunk/splunkd.log

TailingProcessor - Adding watch on path: /logs/sam/ct.
TailingProcessor - Parsing configuration stanza: monitor:///logs/sam/ct/sactws0*/.../web.

[rewritex · Accepted solution]

This is resolved. I had to add the ending /
This works: [monitor:///logs/sam/ct/sact0*/*/web/]

[rewritex]

This is resolved. I had to add the ending /
This works: [monitor:///logs/sam/ct/sact0*/*/web/]

[woodcock]

Yes, otherwise it thinks that you are specifying a file name, not a directory.

[rewritex]

I corrected the whitelist = request.logs*$ to whitelist = request.log*$ to match the naming convention.
Still no luck

[ddrillic]

-- Log files name format within /web (4 months worth, but i only need 7 days):

That goes to best practices of the log directories and archiving. From Splunk perspective, it would be much better if this directory structure contains *only * the 7 days of data. The rest of the files should be moved to an archive area. If possible.

Sifting through a large directory structure and identifying the files to monitor is expensive and the software doesn't give you enough indications where it stands when it processes this directory, so we are in the dark.

[woodcock]

You have told it to search for infinite depth and if you have many subdirectories, Splunk can get WAY too busy. Change it to this:

 [monitor:///logs/sam/ct/sact0*/*/web]