Getting Data In

Is it possible to set monitor paths dynamically without the use of wildcards?

d044160
Explorer

I want to create an app that will be deployed on a few forwarders to monitor log files. Problem is with the paths of these log files. They contain the host name (e.g. /path/<hostname>/logfile.log). I don't want to create an individual app for each host and I can not use wildcards, because defining /path/*/logfile.log in the inputs.conf would bring up a lot of unwanted data - unfortunately, other server's log directories are mounted too. I know that in a configuration stanza I can use $SPLUNK_HOME, so my question is: Is there a way to use variables in the configuration, like


[monitor:///path/$HOSTNAME/logfile.log]

0 Karma

adam_reber
Path Finder

What about using a either the whitelist or blacklist option for the monitor stanza?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Whitelistorblacklistspecificincomingdata

[monitor:///path/*/logfile.log]
blacklist = [Uu]nwanted[Hh]ost[Rr]egex[0-9]/logfile.log$

OR

[monitor:///path/*/logfile.log]
whitelist = /path/goodhost[0-9]/logfile.log$|/path/this[4-9]host/logfile.log$
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Could you use a combination of these?

# Additional attributes:

host_regex = <regular expression>
* If specified, <regular expression> extracts host from the path to the file for each input file. 
    * Detail: This feature examines the source key; if source is set
      explicitly in the stanza, that string will be matched, not the original filename.
* Specifically, the first group of the regex is used as the host. 
* If the regex fails to match, the default "host =" attribute is used.
* If host_regex and host_segment are both set, host_regex will be ignored.
* Defaults to unset.

host_segment = <integer>
* If set to N, the Nth "/"-separated segment of the path is set as host. If host_segment=3, for example,
  the third segment is used.
* If the value is not an integer or is less than 1, the default "host =" attribute is used.
* Defaults to unset.
0 Karma

d044160
Explorer

Very interesting attributes, thanks! But I'm afraid it's the other way round: With those attributes it seems I can extract the host value from the source field, while instead I need to construct a path from a given host. But I'm wondering ... maybe I could approach my problem differently using those attributes -- I guess I'll have to do some experimenting. Thanks for the inspiration though.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Is this answer of any help ?

0 Karma

d044160
Explorer

Unfortunately not. We have a deployment server and we want to deploy the app without having to do additional editing in any of the forwarder's local/inputs.conf.

0 Karma
Get Updates on the Splunk Community!

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...