Getting Data In

Is it possible to set monitor paths dynamically without the use of wildcards?

Explorer

I want to create an app that will be deployed on a few forwarders to monitor log files. Problem is with the paths of these log files. They contain the host name (e.g. /path/<hostname>/logfile.log). I don't want to create an individual app for each host and I can not use wildcards, because defining /path/*/logfile.log in the inputs.conf would bring up a lot of unwanted data - unfortunately, other server's log directories are mounted too. I know that in a configuration stanza I can use $SPLUNK_HOME, so my question is: Is there a way to use variables in the configuration, like


[monitor:///path/$HOSTNAME/logfile.log]

0 Karma

Path Finder

What about using a either the whitelist or blacklist option for the monitor stanza?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Whitelistorblacklistspecificincomingdata

[monitor:///path/*/logfile.log]
blacklist = [Uu]nwanted[Hh]ost[Rr]egex[0-9]/logfile.log$

OR

[monitor:///path/*/logfile.log]
whitelist = /path/goodhost[0-9]/logfile.log$|/path/this[4-9]host/logfile.log$
0 Karma

Splunk Employee
Splunk Employee

Could you use a combination of these?

# Additional attributes:

host_regex = <regular expression>
* If specified, <regular expression> extracts host from the path to the file for each input file. 
    * Detail: This feature examines the source key; if source is set
      explicitly in the stanza, that string will be matched, not the original filename.
* Specifically, the first group of the regex is used as the host. 
* If the regex fails to match, the default "host =" attribute is used.
* If host_regex and host_segment are both set, host_regex will be ignored.
* Defaults to unset.

host_segment = <integer>
* If set to N, the Nth "/"-separated segment of the path is set as host. If host_segment=3, for example,
  the third segment is used.
* If the value is not an integer or is less than 1, the default "host =" attribute is used.
* Defaults to unset.
0 Karma

Explorer

Very interesting attributes, thanks! But I'm afraid it's the other way round: With those attributes it seems I can extract the host value from the source field, while instead I need to construct a path from a given host. But I'm wondering ... maybe I could approach my problem differently using those attributes -- I guess I'll have to do some experimenting. Thanks for the inspiration though.

0 Karma

Splunk Employee
Splunk Employee

Is this answer of any help ?

0 Karma

Explorer

Unfortunately not. We have a deployment server and we want to deploy the app without having to do additional editing in any of the forwarder's local/inputs.conf.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!