Getting Data In

Splunk Indexer down and bring up after 1 days passed, Do the forwarder continue forwarding log since Indexer down?

New Member

I have 2 Splunk Indexer and have 10 Splunk Forwarder forward to both Splunk Indexer. Once Indexer01 down for 1 day and Indexer01 bring up to online again then all forwarder will be re-forwarding log since Indexer01 down or not?
If not, Indexer01 will loss data 1 day right?

In the mean time all Indexer back to running as normal, Indexer01 indexing data will less than Indexer02 right?

Do we have solution to sync data between Indexer01 and Indexer02? Thanks.

0 Karma

Influencer

How do you have your deployment configured? - are you using Splunk's autoLB or are you simply forwarding to both servers?

You should read the documentation on forwarding and receiving, specifically... THIS
If you are using autoLB, then if Indexer01 goes offline for some reason then the forwarders will continue to forward the data to Indexer02. If you are using this set-up then you will must likely be using a distributed search across both indexers so you shouldn't "see" any disruption in the search results.

If you are forwarding to both indexers, and Indexer01 goes offline, the data will continue to be forwarded to the second indexer, when Indexer01 is bought back online it will continue to forward data that has been cached as it could not make a TCP connection. Obviously depending on the amount of data you index daily, you may have to wait for it to "catch up". So until Indexer01 has caught up it will have less data.

If you wanted to duplicate the data on both indexers, this will most likely incur charges on your license usage (due to re-indexing), unless you try something like rsync, but not sure that Splunk would allow that on the indexes (i.e. not tried it myself).