Getting Data In

Is it possible to set monitor paths dynamically without the use of wildcards?

d044160
Explorer

I want to create an app that will be deployed on a few forwarders to monitor log files. Problem is with the paths of these log files. They contain the host name (e.g. /path/<hostname>/logfile.log). I don't want to create an individual app for each host and I can not use wildcards, because defining /path/*/logfile.log in the inputs.conf would bring up a lot of unwanted data - unfortunately, other server's log directories are mounted too. I know that in a configuration stanza I can use $SPLUNK_HOME, so my question is: Is there a way to use variables in the configuration, like


[monitor:///path/$HOSTNAME/logfile.log]

0 Karma

adam_reber
Path Finder

What about using a either the whitelist or blacklist option for the monitor stanza?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Whitelistorblacklistspecificincomingdata

[monitor:///path/*/logfile.log]
blacklist = [Uu]nwanted[Hh]ost[Rr]egex[0-9]/logfile.log$

OR

[monitor:///path/*/logfile.log]
whitelist = /path/goodhost[0-9]/logfile.log$|/path/this[4-9]host/logfile.log$
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Could you use a combination of these?

# Additional attributes:

host_regex = <regular expression>
* If specified, <regular expression> extracts host from the path to the file for each input file. 
    * Detail: This feature examines the source key; if source is set
      explicitly in the stanza, that string will be matched, not the original filename.
* Specifically, the first group of the regex is used as the host. 
* If the regex fails to match, the default "host =" attribute is used.
* If host_regex and host_segment are both set, host_regex will be ignored.
* Defaults to unset.

host_segment = <integer>
* If set to N, the Nth "/"-separated segment of the path is set as host. If host_segment=3, for example,
  the third segment is used.
* If the value is not an integer or is less than 1, the default "host =" attribute is used.
* Defaults to unset.
0 Karma

d044160
Explorer

Very interesting attributes, thanks! But I'm afraid it's the other way round: With those attributes it seems I can extract the host value from the source field, while instead I need to construct a path from a given host. But I'm wondering ... maybe I could approach my problem differently using those attributes -- I guess I'll have to do some experimenting. Thanks for the inspiration though.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Is this answer of any help ?

0 Karma

d044160
Explorer

Unfortunately not. We have a deployment server and we want to deploy the app without having to do additional editing in any of the forwarder's local/inputs.conf.

0 Karma
Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...