Universal Forwarder Not Monitoring Directory

[sjnorman]

I am having issues setting up a UNIX universal forwarder to monitor IBM IHS http log files -- it does not appear to be setting up the monitor. I can see that the stanza in the inputs.conf is being parsed by looking at the splunkd.log, but I don't see a corresponding entry "Adding watch on path...".

Am I missing something in my configuration? What would prevent a watch from being added? I've checked the UNIX file permissions and the 'splunk' user has read-access to all files and directories.

All other monitored entities in my inputs.conf have a log entry indicating that a watch has been started. There are no errors in the splunkd.log file.

splunkd.log on forwarder host

TailingProcessor - Parsing configuration stanza: monitor:///apps/logs/http/IBMIHS.
...
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/logs/WebSphere/AS_WAS_01.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/logs/WebSphere/AS_WAS_02
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/etc/splunk.version.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/var/log/splunk.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/var/spool/splunk.

--> no corresponding entry saying that a watch has been added for IHS -- see config below.

inputs.conf on forwarder host

[monitor:///apps/logs/http/IBMIHS]
disabled = false
recursive = false
index = myindex
blacklist = \.(gz)$
sourcetype = access_combined

inputs.conf on indexer host

[access_combined]
pulldown_type = true 
maxDist = 28
MAX_TIMESTAMP_LOOKAHEAD = 128
REPORT-access = access-extractions
SHOULD_LINEMERGE = False
TIME_PREFIX = \[

transforms.conf on indexer host

[access-extractions] 
\# matches access-common or access-combined apache logging formats 
\# Extracts: clientip, clientport, ident, user, req_time, method, uri, root, file, uri_domain, uri_query, version, status, bytes, referer_url, referer_domain, referer_proto, useragent, cookie, other (remaining chars)   
\# Note: referer is misspelled in purpose because that is the "official" spelling for "HTTP referer"  
REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]](?:\s++"(?<referer>[[bc_domain:referer_]]?+[^"]*+)"(?:\s++[[qstring:useragent]](?:\s++[[qstring:cookie]])?+)?+)?[[all:other]]

[sjnorman]

Used workaround outlined in here

[tgow]

Everything looks good. The blacklist is strange though. Do you really need the parentheses around gz? I have always done it:

blacklist=\.gz$

[sjnorman]

I've adjusted the blacklist parameters & re-started the universal forwarder.

Sill no go...no log entry that indicates that a watch has been added, nor is the indexer picking anything up.