Getting Data In

Is it possible to set monitor paths dynamically without the use of wildcards?

d044160
Explorer

I want to create an app that will be deployed on a few forwarders to monitor log files. Problem is with the paths of these log files. They contain the host name (e.g. /path/<hostname>/logfile.log). I don't want to create an individual app for each host and I can not use wildcards, because defining /path/*/logfile.log in the inputs.conf would bring up a lot of unwanted data - unfortunately, other server's log directories are mounted too. I know that in a configuration stanza I can use $SPLUNK_HOME, so my question is: Is there a way to use variables in the configuration, like


[monitor:///path/$HOSTNAME/logfile.log]

0 Karma

adam_reber
Path Finder

What about using a either the whitelist or blacklist option for the monitor stanza?

http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Specifyinputpathswithwildcards
http://docs.splunk.com/Documentation/Splunk/6.2.1/Data/Whitelistorblacklistspecificincomingdata

[monitor:///path/*/logfile.log]
blacklist = [Uu]nwanted[Hh]ost[Rr]egex[0-9]/logfile.log$

OR

[monitor:///path/*/logfile.log]
whitelist = /path/goodhost[0-9]/logfile.log$|/path/this[4-9]host/logfile.log$
0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Could you use a combination of these?

# Additional attributes:

host_regex = <regular expression>
* If specified, <regular expression> extracts host from the path to the file for each input file. 
    * Detail: This feature examines the source key; if source is set
      explicitly in the stanza, that string will be matched, not the original filename.
* Specifically, the first group of the regex is used as the host. 
* If the regex fails to match, the default "host =" attribute is used.
* If host_regex and host_segment are both set, host_regex will be ignored.
* Defaults to unset.

host_segment = <integer>
* If set to N, the Nth "/"-separated segment of the path is set as host. If host_segment=3, for example,
  the third segment is used.
* If the value is not an integer or is less than 1, the default "host =" attribute is used.
* Defaults to unset.
0 Karma

d044160
Explorer

Very interesting attributes, thanks! But I'm afraid it's the other way round: With those attributes it seems I can extract the host value from the source field, while instead I need to construct a path from a given host. But I'm wondering ... maybe I could approach my problem differently using those attributes -- I guess I'll have to do some experimenting. Thanks for the inspiration though.

0 Karma

aljohnson_splun
Splunk Employee
Splunk Employee

Is this answer of any help ?

0 Karma

d044160
Explorer

Unfortunately not. We have a deployment server and we want to deploy the app without having to do additional editing in any of the forwarder's local/inputs.conf.

0 Karma
Get Updates on the Splunk Community!

Announcing Scheduled Export GA for Dashboard Studio

We're excited to announce the general availability of Scheduled Export for Dashboard Studio. Starting in ...

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics GA in US-AWS!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...