Getting Data In

Is it possible to set monitor paths dynamically without the use of wildcards?

d044160
Explorer

I want to create an app that will be deployed on a few forwarders to monitor log files. Problem is with the paths of these log files. They contain the host name (e.g. /path/<hostname>/logfile.log). I don't want to create an individual app for each host and I can not use wildcards, because defining /path/*/logfile.log in the inputs.conf would bring up a lot of unwanted data - unfortunately, other server's log directories are mounted too. I know that in a configuration stanza I can use $SPLUNK_HOME, so my question is: Is there a way to use variables in the configuration, like


[monitor:///path/$HOSTNAME/logfile.log]

0 Karma

adam_reber
Path Finder

What about using a either the whitelist or blacklist option for the monitor stanza?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Whitelistorblacklistspecificincomingdata

[monitor:///path/*/logfile.log]
blacklist = [Uu]nwanted[Hh]ost[Rr]egex[0-9]/logfile.log$

OR

[monitor:///path/*/logfile.log]
whitelist = /path/goodhost[0-9]/logfile.log$|/path/this[4-9]host/logfile.log$
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Could you use a combination of these?

# Additional attributes:

host_regex = <regular expression>
* If specified, <regular expression> extracts host from the path to the file for each input file. 
    * Detail: This feature examines the source key; if source is set
      explicitly in the stanza, that string will be matched, not the original filename.
* Specifically, the first group of the regex is used as the host. 
* If the regex fails to match, the default "host =" attribute is used.
* If host_regex and host_segment are both set, host_regex will be ignored.
* Defaults to unset.

host_segment = <integer>
* If set to N, the Nth "/"-separated segment of the path is set as host. If host_segment=3, for example,
  the third segment is used.
* If the value is not an integer or is less than 1, the default "host =" attribute is used.
* Defaults to unset.
0 Karma

d044160
Explorer

Very interesting attributes, thanks! But I'm afraid it's the other way round: With those attributes it seems I can extract the host value from the source field, while instead I need to construct a path from a given host. But I'm wondering ... maybe I could approach my problem differently using those attributes -- I guess I'll have to do some experimenting. Thanks for the inspiration though.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Is this answer of any help ?

0 Karma

d044160
Explorer

Unfortunately not. We have a deployment server and we want to deploy the app without having to do additional editing in any of the forwarder's local/inputs.conf.

0 Karma
Get Updates on the Splunk Community!

Automatic Discovery Part 1: What is Automatic Discovery in Splunk Observability Cloud ...

If you’ve ever deployed a new database cluster, spun up a caching layer, or added a load balancer, you know it ...

Real-Time Fraud Detection: How Splunk Dashboards Protect Financial Institutions

Financial fraud isn't slowing down. If anything, it's getting more sophisticated. Account takeovers, credit ...

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...