Deployment Architecture

Re-indexing zip files when Windows Cluster failover impact to duplicate logs forwarded.

Explorer

Windows Cluster machine [Node A - Active/ Node B - Stand by] and use SAN for central storage.
All Splunk configuration set as default except input.conf which modified to monitoring specific log path.
We have installed Splunk Forwarder 4.2 on both node and monitor file.zip on SAN and NAS path. Application created "Splunk_DB" on SAN for tracking log forwarded. All log files monitored are working as expected except file.zip have a problem about duplicate log file forwarded when Cluster switch node. All log which forwarded on Node A will re-indexing when active node switch to Node B.

Impact: Forwarder re-indexing duplicated content on Splunk Indexer and license limit exceed.

Need urgently long term solution from Splunk support team. Thank you in advance.

Related question : http://splunk-base.splunk.com/answers/43531/failover-cluster-splunk-re-index-when-cluster-has-switch...

0 Karma

Explorer

When the major release will be launch?

0 Karma

Builder

Hello,
This is a known bug in splunk. bug SPL-39144. It is likely due to the fact that the indexed position in a zip file is not properly tracked. Unfortunately I don't have a good ETA on a fix for you. It is at least one major release away from being addressed. As a workaround, i suggest monitoring the files before they are zipped or setup a script to decompress the zip files into another directory that splunk can monitor.

Explorer

When we have a solution for this problem?

0 Karma