I have Splunk Cloud and I need to ingest data from a database. I have this working fine in a lab - with one standalone Splunk box using DB Connect.
I'd like to deploy this in a resilient configuration for a customer. We're installing a pair of Heavy Forwarders to ensure resilience for syslog and UF inputs, but how can I achieve a resilient configuration for my DB Connect source?
DB Connect stores the most recent rising column value in an input-specific file under /opt/splunk/var/lib/splunk/modinputs/server/splunk_app_db_connect. Surely the only way of making this work would be for this value to be shared across two hosts running DBConnect?..
Know this is an old question and I'm assuming a solution may have been found by now, but given that I had to deal with a similar situation recently, I thought I'd add the solution we put in place.
We had 2 servers which were set-up as a high availability cluster, which from a Splunk point of view is OK - which ever server is running will forward the logs, but as mentioned, with DB connect there is the rising column files to factor in.
Solution for us here was to have a floating drive set-up that would move with the active node. We installed Splunk onto this drive.
Yes, using full Splunk Enterprise we completed the install on one node into the floating drive first.
Then fail over so the floating drive moved to newly active node and installed again (to same location on floating drive).
Then just update inputs.conf and server.conf with an entry for host / server name that makes sense - will be the same no matter which node is active.
You just have to make sure that Splunk doesn't try to start before the floating drive has been made available to the node.
Currently there is no way to have HA with Splunk + DBX internally.
The best way our customers have approached this is to use some form of OS based clustering, such as Red Hat Clustering etc. Then you can have the HA based at the OS level...