Getting Data In

Universal Forwarder Not Monitoring Directory

sjnorman
Explorer

I am having issues setting up a UNIX universal forwarder to monitor IBM IHS http log files -- it does not appear to be setting up the monitor. I can see that the stanza in the inputs.conf is being parsed by looking at the splunkd.log, but I don't see a corresponding entry "Adding watch on path...".

Am I missing something in my configuration? What would prevent a watch from being added? I've checked the UNIX file permissions and the 'splunk' user has read-access to all files and directories.

All other monitored entities in my inputs.conf have a log entry indicating that a watch has been started. There are no errors in the splunkd.log file.

splunkd.log on forwarder host

TailingProcessor - Parsing configuration stanza: monitor:///apps/logs/http/IBMIHS.
...
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/logs/WebSphere/AS_WAS_01.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/logs/WebSphere/AS_WAS_02
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/etc/splunk.version.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/var/log/splunk.
09-11-2014 08:53:48.274 -0500 INFO  TailingProcessor - Adding watch on path: /apps/splunkforwarder/var/spool/splunk.

--> no corresponding entry saying that a watch has been added for IHS -- see config below.

inputs.conf on forwarder host

[monitor:///apps/logs/http/IBMIHS]
disabled = false
recursive = false
index = myindex
blacklist = \.(gz)$
sourcetype = access_combined

inputs.conf on indexer host

[access_combined]
pulldown_type = true 
maxDist = 28
MAX_TIMESTAMP_LOOKAHEAD = 128
REPORT-access = access-extractions
SHOULD_LINEMERGE = False
TIME_PREFIX = \[

transforms.conf on indexer host

[access-extractions] 
\# matches access-common or access-combined apache logging formats 
\# Extracts: clientip, clientport, ident, user, req_time, method, uri, root, file, uri_domain, uri_query, version, status, bytes, referer_url, referer_domain, referer_proto, useragent, cookie, other (remaining chars)   
\# Note: referer is misspelled in purpose because that is the "official" spelling for "HTTP referer"  
REGEX = ^[[nspaces:clientip]]\s++[[nspaces:ident]]\s++[[nspaces:user]]\s++[[sbstring:req_time]]\s++[[access-request]]\s++[[nspaces:status]]\s++[[nspaces:bytes]](?:\s++"(?<referer>[[bc_domain:referer_]]?+[^"]*+)"(?:\s++[[qstring:useragent]](?:\s++[[qstring:cookie]])?+)?+)?[[all:other]]
0 Karma

sjnorman
Explorer

Used workaround outlined in here

0 Karma

tgow
Splunk Employee
Splunk Employee

Everything looks good. The blacklist is strange though. Do you really need the parentheses around gz? I have always done it:

blacklist=\.gz$
0 Karma

sjnorman
Explorer

I've adjusted the blacklist parameters & re-started the universal forwarder.

Sill no go...no log entry that indicates that a watch has been added, nor is the indexer picking anything up.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...