topic Re: Expand inputs.conf with wildcards in Getting Data In

Expand inputs.conf with wildcards

tyronetv — Tue, 12 Nov 2013 21:15:48 GMT

Does anyone know of a tool that will 'expand' the monitor stanza from inputs.conf on a universalforwarder to show an example of logs to be watched?

I.e., I have a monitor stanza:

[monitor:///path/to/some/*/dir]
whitelist = /file_name(s).log$

And before I restart splunk and do the 'hope it works' I was wondering if there was a tool that would, using Splunk's logic, show me all the files the above would 'see' for monitoring.

I have multiple 'client' directories (being replaced above by the *) where some have specific logs and some do not. I would rather write one monitor for each type of log verses writing a new monitor stanza per client dir/log type.

And I need to test it before pulling the trigger and not impact other, already configured, data-gathering.

Re: Expand inputs.conf with wildcards

jtrucks — Tue, 12 Nov 2013 21:38:26 GMT

A fairly simplistic approach is just to use ls:

ls -d /path/to/some/*/dir
ls -d /path/to/some/*/dir/file_name*.log

The results is how the system will glob the filenames and create paths.

Also, you could quickly write something in perl, python, C, or any other language with a similar function. Then you could have that program pull any line with "[monitor…]" to parse the paths and glob them for you.

For a working way to do this really quick and dirty, do this:

ls -d $( awk '/monitor/' inputs.conf| sed -e 's|\[monitor://||' -e 's|\]$||')

Obviously adjust where you run this or specify full path to inputs.conf.

Re: Expand inputs.conf with wildcards

jtrucks — Tue, 12 Nov 2013 22:04:33 GMT

I assume the poster downvoted me because I didn't provide a ready to use answer, so now there is one. Please upvote it and accept as working if you test this and it works.

Thanks.

Re: Expand inputs.conf with wildcards

tyronetv — Tue, 12 Nov 2013 22:38:44 GMT

The awk statement is fine and almost a mirror of what I've already done. I am looking for something that essentially mimics the expansion of the entire monitor stanza to include file names identified by the white/black lists as well as the monitor line.

Re: Expand inputs.conf with wildcards

jtrucks — Tue, 12 Nov 2013 22:42:26 GMT

There isn't a premade tool that does that to date that anyone has published. It might make a good feature request to Splunk.