Splunk Forwarder Unix App not sending logs

[brianokelly]

We installed the unix app on our universal forwarder on RHEL6. We modified the inputs.conf to the following:

[monitor:///var/log]
_whitelist=(\.log|log$|^messages|^secure|mesg$|cron$|acpid$|\.out)
_blacklist=(lastlog)
index=os
disabled = 0

We noticed though that we did not receive any logs from /var/log/messages even though it appears in the list. We did however receive other log messages. We had to add the following for it to correctly monitor the messages file:

[monitor:///var/log/messages] 
index = os 
disabled = 0

Has anyone seen a reason as to why this could happen.

[davewood]

I had the same issue. Sometime between 4.2 and 4.2.2, whitelist has seemingly been changed to match the entire path, not just the filename. This means the the "^" anchor prevents "/var/log/messages" being matched, so if you change the regex to be something like:

whitelist=(\.log|log$|\/messages|\/secure|mesg$|cron$|acpid$|\.out)

(i.e. match a filename containing "/messages")

it should work.

If you're being pedantic, then you may notice this will also match something like:

/var/log/someapp/messages/not_a_logfile

So if you're worried about that then you could use the following pattern to match exactly the files that would have been matched previously:

whitelist=(\.log|log$|\/messages[^\/]*$|\/secure[^\/]*$|mesg$|cron$|acpid$|\.out)

(i.e. a filename beginning with "messages").

[Drainy]

Couldn't you just incorporate it as |messages| in the regex? this will be an exact match then (Unless you've already tried this 🙂

Also, whilst not a really a problem the _whitelist and _blacklist are depreciated and should really be replaced with just whitelist and blacklist (assuming you're using a fairly new version of the UF.
http://docs.splunk.com/Documentation/Splunk/latest/admin/inputsconf

Alternatively you could just use the blacklist filter to just filter whichever logs you don't want it to index and don't define a whitelist at all.