Deployment Architecture

Deployment Server: Support Dev & Prod enviroments without duplicating inputs.conf

krussell101
Path Finder

I have a single Indexer serving everything: production and test.

I've defined two servers classes: production and test. (I'm simplifying my world to better highlight my questions.)

I have two applications running in both test and production called "gateway" and "messagebroker". In prod they are running on their own servers. In test they share a server.

What I want to be able to do but don't know how:

QUESTION 1) Have a single outputs.conf file. I only have one indexer so I don't want multiple outputs.conf files. Where do I put this one outputs.conf file?

QUESTION 2) For production hosts I want the default index to be "prod" with some sources going to "prodsyst", likewise "test" and "testsyst" for test hosts. So I'd like an inputs.conf file for production hosts that looks like this:

[global]
index = test

[monitor:///var/log]
index = testsyst

In the specific app inputs.conf I will not define indexes. Is this possible and if so where do I put these two inputs.conf files?

QUESTION 3) For every instance of the "messagebroker" app I want to add specific directories to monitor that will be same for both prod and test, thus only a single inputs.conf file for messagebroker. In short, I don't want a serverclass file that includes [serverClass:productionservers:app:messagebroker] and [serverClass:testservers:app:messagebroker].

I really appreciate your help. I have read the splunk documentation and understand their examples, but their examples don't seem to cover this. Furthermore, it's the not the creation of the serverclass.conf file that puzzles me. It's what directories to create on the deployment server to hold my various config files. THANKS!

Here is my serverclass.conf on my deployment server:


[global]
whitelist.0=*

[serverClass:linuxservers]
whitelist.0 = *.*.mycompany.inc

[serverClass:productionservers]
filterType = whitelist
whitelist.0 = *.prod.mycompany.inc

[serverClass:testservers]
filterType = whitelist
whitelist.0 = *.test.mycompany.inc

[serverClass:linuxservers:app:gateway]
stateOnClient=enabled
restartSplunkd=true

[serverClass:linuxservers:app:messagebroker]
stateOnClient=enabled
restartSplunkd=true

Tags (1)
1 Solution

dwaddle
SplunkTrust
SplunkTrust

Question #1 -- my advice would be an app specific to outputs.conf. As your Splunk deployment evolves, you may have many other servers running dozens of applications all running forwarders. The commonality of all of these is they need to be able to tie into your indexers. So, it makes sense to build a mydotcom-forwarderconfig app that deploys outputs.conf, limits.conf, and whatever else you need that is "standard" forwarder config.

View solution in original post

dwaddle
SplunkTrust
SplunkTrust

Question #1 -- my advice would be an app specific to outputs.conf. As your Splunk deployment evolves, you may have many other servers running dozens of applications all running forwarders. The commonality of all of these is they need to be able to tie into your indexers. So, it makes sense to build a mydotcom-forwarderconfig app that deploys outputs.conf, limits.conf, and whatever else you need that is "standard" forwarder config.

View solution in original post

krussell101
Path Finder

Cool. Think I have it now. Thanks for the all the help!

0 Karma

sowings
Splunk Employee
Splunk Employee

Yes, they are additive. To avoid unexpected collisions, you may want to place your indexes.conf in your thisismyindexapp's local subdirectory, acting as though it were an override to the config. This means that thisismyindexapp's edits will take precedence over defaults, whether system wide or application-level. Note that configs in system/local are like trump cards--they override even items in application local directories.

krussell101
Path Finder

Got it.

I suspect I'm thinking of "app" too literally. So your suggestion could also be the answer to the rest of my questions.

I could have an serverclass:productionservers:app:thisismyindexapp which define the indexes for all production servers.

Configuration files are additive too, correct? I can have an inputs.conf in both productionservers:app:thisismyindexapp and linuxservers:app:messagebroker?

0 Karma

dwaddle
SplunkTrust
SplunkTrust

In your example, it would belong to the linuxservers class, or an everyone class that represents ALL of the forwarders. Remember, classes are additive - a host can be a member of many classes.

0 Karma

krussell101
Path Finder

Apps have to belong to serverclasses as I understand it. What serverclass would this app belong to?

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!