RSyslog, Dynamic Filenames, and Host_Regex

[milesbrennan]

Hi Splunkers,

We're using Rsyslog to collect many of our appliance syslog streams, and then bringing them into Splunk on our heavy forwarder.

However the issue we have is all "host" entries are using the heavy forwarder hostname, and not the syslog/appliance hostname.

We've adjusted our Rsyslog configuration file to use dynamic filenames, with the following configuration:

#  vi /etc/rsyslog.d/splunk.conf
template (name="websense-proxy" type="string" string="/var/log/splunk/websense/%HOSTNAME%.log")
if $programname == "vendor=Forcepoint" and $msg contains "product=Security" then { action (type="omfile" dynafile="websense-proxy") stop }

This works fine, and the log files are appearing correctly as:

/var/log/splunk/websense/proxy01.log
/var/log/splunk/websense/proxy02.log

However our "host_regex" does not work properly, and should be picking up the filename as the hostname (minus the file extention).

[monitor://var/log/splunk/websense/.*log]
index = proxy
sourcetype = websense
host_regex = .*\/(.+)\.log

We've gone through the inputs.conf / host_regex documentation and several posts here, but can't get it working with Splunk properly.

However it seems to be working fine in Regex101 - https://regex101.com/r/ayZak8/3

In Splunk we're expecting to see host proxy01 and proxy02.

[milesbrennan · Accepted solution]

I was able to get this working after changing monitor stanza to three "///" i.e.

This works:

[monitor:///var/log/splunk/websense-*log]
index = proxy
sourcetype = websense
host_regex = .*\-(.+)\.log

However using a directory continues to fail:

[monitor:///var/log/splunk/websense/*log]
index = proxy
sourcetype = websense
host_regex = .*\/(.+)\.log

So I'm forced to place all the logs into the main directory, I can't move them into their own "sourcetype" subdirectories - not a major issue, however it would be a little neater and help my OCD.... 😉

The main point however, is the host_regex is now working on the "*" wildcard, and hostnames are updating correctly in Splunk

So in /etc/rsyslog.d/splunk.conf, we have the declarations like these, which makes the filtering more dynamic:

template (name="fortigate"    type="string" string="/var/log/splunk/fortigate-%HOSTNAME%.log")
template (name="fortiweb"     type="string" string="/var/log/splunk/fortiweb-%HOSTNAME%.log")
template (name="fortiwebcef"  type="string" string="/var/log/splunk/fortiwebcef-%HOSTNAME%.log")
template (name="mfa"          type="string" string="/var/log/splunk/mfa-%HOSTNAME%.log")


if $msg contains "devid=FGT" then           { action (type="omfile" dynafile="fortigate") stop }
if $msg contains "device_id=FVVM" then      { action (type="omfile" dynafile="fortiweb") stop }
if $programname == "CEF" and $msg contains "FortiWeb" then       { action (type="omfile" dynafile="fortiwebcef") stop }
if $programname == "pfsvc" then             { action (type="omfile" dynafile="mfa") stop }

Thanks for all posts / answers.

Cheers

[sbbadri]

@milesbrennan
try below,
[monitor://var/log/splunk/websense/*.log]
index = proxy
sourcetype = websense
host_regex =/var/log/splunk/websense/(.+).log

[adonio]

Hello there,

will recommend to use slightly different method.
have the rsyslog configured to have the host name in a directory path, example:

/var/log/splunk/websense/<hostName>/*.log

now you can use the wonderful host_segment conf
in this case, inputs.conf will look like that:

 [monitor://var/log/splunk/websense/.../*.log]
  index = proxy
  sourcetype = websense
  host_segment = 5

and you are ready to rock and roll

hope it helps

[jkat54]

.* does not work this way in a monitor stanza. As the documentation of inputs.conf says:

Note concerning wildcards and monitor:
* You can use wildcards to specify your input path for monitored input. Use
  "..." for recursive directory matching and "*" for wildcard matching in a
  single directory segment.
* "..." recurses through directories. This means that /foo/.../bar will match
  foo/bar, foo/1/bar, foo/1/2/bar, etc.
* You can use multiple "..." specifications in a single input path. For
  example: /foo/.../bar/...
* The asterisk (*) matches anything in a single path segment; unlike "...", it
  does not recurse. For example, /foo/*/bar matches the files /foo/bar,
  /foo/1/bar, /foo/2/bar, etc. However, it does not match /foo/1/2/bar.
  A second example: /foo/m*r/bar matches /foo/mr/bar, /foo/mir/bar,
  /foo/moor/bar, etc.
* You can combine "*" and "..." as needed: foo/.../bar/* matches any file in
  the bar directory within the specified path.

Therefore, this will work

 [monitor://var/log/splunk/websense/*.log]
 index = proxy
 sourcetype = websense
 host_regex = .*\/(.+)\.log

[milesbrennan]

I've progressed a little further on this, it fails to work when I use a wildcard "*" in the monitor stanza, and works perfectly when I use an absolute path / filename.

This fails:

[monitor://var/log/splunk/websense/.*log]
index = proxy
sourcetype = websense
host_regex = .*\/(.+)\.log

But this works:

[monitor://var/log/splunk/websense/proxy01.log]
index = proxy
sourcetype = websense
host_regex = .*\/(.+)\.log

This indexes host=proxy01

However this means I would need a unique stanza for every single file I want to index, which doesn't make sense, surely this can be more dynamic.