allowedHosts option in inputs.conf

[davidstuffle]

I'm looking for a way to specify which host a [monitor] stanza applies to within the inputs.conf file. If the stanza isn't on an "allowedHost", it will be ignored. Something like this, for example:

[monitor:///nfs_share/app1/logs]
allowedHosts=linux_server1

[monitor:///nfs_share/app2/logs]
allowedHosts=linux_server2

[monitor://D:\logs]
allowedHosts=windows_server1

This way I can apply this one inputs.conf file to several servers where some of the paths may not be valid, or wanted, and it will simply ignore the stanza if it's not on one of the allowed hosts.

Does this already exist is some other form? I know of some less than elegant ways to do this.

I understand one option is to just let it error out if a path is not valid...but sometimes the path IS valid, but I don't want it included unless we're on the correct server, as in the NFS share example.

[gabmsmith]

I was looking to figure out how to do this too and searched all through the inputs.conf documentation and couldn't find what I was looking for. Would be cool and useful.

[PickleRick]

As already stated, splitting inputs into separate apps and associating them with different serverclasses is the way to go.

An input is a relatively "simple" idea. It might have features letting you filter _what_ you're ingesting (like particular files or windows event ids) but not _where_ they run or not.

[MikaJustasACN]

I am facing the same issue. What if you have 100 different host and with same log type. On each server the log name is completely different and the path is completely different not allowing wildcards, whats then? What is the approach? I cannot do /.../*.log, that's too much and creating 100 serverclasses, well, I would rather not.

It is not a rare case I believe, thus what would be the suggested approach?

[ddrillic]

Then something seems to be fundamentally wrong with the log files organization. If you can't change it, this approach might make sense.

[sloshburch]

@MikaJustasACN - I wonder if there's more to the context than we have talked about here. I actually have not seen many customers where there is such aggressive diversity in paths. Organizations I've talked to have always either had standard deployment patterns with standard log location - albeit sometimes with variable portions of the paths. The reason such a pattern exists has been to ensure the logs are discover-able by supporting engineers.

For example, in my prior life, we would condense web servers on the same hardware but have different installs for each web server (and therefore different logs). But this also meant we had a convention for how we performed these parallel installs so we could differentiate between them and their logs. Such patterns were easily reproduced in our inputs.conf and then pushed out with our serverclass.conf.

I invite you to share more details or examples on the challenge you face since perhaps we are just overlooking or misunderstanding the critical nuances.

[MikaJustasACN]

I am a bit exaggerating here of course to make the point. My real life scenario is 30 servers with approx 25 different paths where using wildcards (... and *) I will fit into probably 6 different applications. Found this thread hoping there something introduced in Splunk like david raised 🙂 I spoke with app owners, its something legacy and they are not willing to consolidate the logs into standard paths. But then I was thinking, can we do symlinks to a single path and pull symlink?

[sloshburch]

I think you can, but having it come down to about 6 paths sounds manageable... Symlinks have their own risks and fragility that might not make them worth it.

You're welcome to post obfuscated versions of the paths and take advantage of our outside perspective to see if we catch something else that's possible.

[adonio]

why not create unique serverclasses for the servers and have a small input deployment app for each?

[davidstuffle]

Hi adonio...Unique serverclasses is the way we are currently doing it. It just results in a lot of serverclasses to deal with and seems to me that it defeats the purpose of serverclasses if we have a unique one for each host.

[sloshburch]

...defeats the purpose of serverclasses if we have a unique one for each host.

That sounds like a potential oversight or misunderstanding on how the serverclasses can be defined. You may have many/multiple hosts in your serverclass definition. You can even use wildcards or some OS level matches. AND since they use the clientName, not the hostname, you can even grow to create your own naming convention and pattern to embrace further automation.

This page of the manual might be where you want to revisit: Set up client filters

Of course, revert back if you have any questions! Happy Splunking!

[davidstuffle]

Hey Burch. I can add multiple hosts to a serverclass, but when there is a specific path to monitor that is unique per server, it forces me to create a unique serverclass for each host. host_regex and host_segment within the inputs doesn't help in this instance either. Maybe next time you are on-site we can talk about it. I'll just leave this as a feature request for now...I was curious if anyone else would find it useful to have.

[sloshburch]

Oh! Well then that's where wildcards come in within the inputs.conf!

/path/on/server1/to/logs.log and /path/on/server2/to/logs.log could be one monitor stanza in the inputs.conf:
/path/on/server*/to/logs.log
Learn more at: http://docs.splunk.com/Documentation/Splunk/latest/Data/Specifyinputpathswithwildcards

Does that work for you? Or have I misunderstood again?