Where do I configure inputs.conf for Splunk DB Connect 1.1.5 in a search head pooling environment?

[bohrasaurabh]

We faced HTTP 401 issues with Java Bridge for DB Connect 1.1.6, so I downgraded it to 1.1.5 and the bridge started right away.

We have a Search Head Pooling environment and I followed the Documentation to configure DB Connect and verified that I can see the database configured on all servers.

http://docs.splunk.com/Documentation/DBX/1.1.5/DeployDBX/Setupsearchheadpooling

Per documentation "While you can install and configure Splunk DB Connect on a single search head, in a pooling environment, the app state is written to shared storage and is visible to all search heads. "

I want to know where should I place my inputs.conf based on the above statement?

When I put it in $SPLUNK_HOME/etc/system/local/ , I get below message

2015-01-22 12:32:49.281 monsch1:INFO:Scheduler - No database input configured. No need to start dbmon scheduler.
2015-01-22 12:32:49.281 monsch1:INFO:Scheduler - Shutting down database inputs...
2015-01-22 12:32:49.282 monsch1:INFO:ExecutionContext - Execution finished in duration=615 ms
2015-01-22 12:37:48.448 dbx6340:INFO:Splunkd - Splunkd REST Keep-alive successful for user splunk-system-user
2015-01-22 12:37:48.448 dbx6340:INFO:ExecutionContext - Execution finished in duration=13 ms

The btool in the inputs finds my stanza prefix.

[bohrasaurabh · Accepted solution]

Based on inputs from Splunk PS and testing, in the SHP environment the inputs.conf for DBConnect should be on $SPLUNK_HOME/etc /system/local area on ONLY 1 SH.

When I had asked this question, I had already put inputs.conf with just dbmon-tail in the local area and restarted splunk, however the inputs.conf was not getting read and the above messages were getting logged. After several retries, when this did not work, I added another input - with dbmon-dump. Once I did that and restarted splunk, suddenly inputs.conf in system/local area started getting read and DBconnect started working - HOW -WHY, I don't know.

I had opened a ticket with support to validate this, however per them inputs.conf for DB connect outside pooled area is not supported but they were also unable to tell me how to configure DB Connect based on documentation statement: "While you can install and configure Splunk DB Connect on a single search head, in a pooling environment, the app state is written to shared storage and is visible to all search heads. ". and kept referring to the Architecture statement:

`About search head pooling and dbmon-tail

We do not recommend using dbmon-tail inputs in a search head pooling environment. In a search head pooling environment, each search head has its own persistent storage that keeps track of the last rising column. This can cause Splunk to index different values for each search head.

We recommend instead that you use a dedicated heavy forwarder with DB Connect installed, to forward data to Splunk indexers.`

Anyways, at this point DB connect has been working for us over a month as expected in the SHP environment and I have not seen any performance issue on any of the SH's.

[abhijitmishra]

Place it in ~SearhHeadPool/etc/apps/dbx/local