Deployment Architecture

Re-indexing zip files when Windows Cluster failover impact to duplicate logs forwarded.

TRAAAL3
Explorer

Windows Cluster machine [Node A - Active/ Node B - Stand by] and use SAN for central storage.
All Splunk configuration set as default except input.conf which modified to monitoring specific log path.
We have installed Splunk Forwarder 4.2 on both node and monitor file.zip on SAN and NAS path. Application created "Splunk_DB" on SAN for tracking log forwarded. All log files monitored are working as expected except file.zip have a problem about duplicate log file forwarded when Cluster switch node. All log which forwarded on Node A will re-indexing when active node switch to Node B.

Impact: Forwarder re-indexing duplicated content on Splunk Indexer and license limit exceed.

Need urgently long term solution from Splunk support team. Thank you in advance.

Related question : http://splunk-base.splunk.com/answers/43531/failover-cluster-splunk-re-index-when-cluster-has-switch...

0 Karma

TRAAAL3
Explorer

When the major release will be launch?

0 Karma

RicoSuave
Builder

Hello,
This is a known bug in splunk. bug SPL-39144. It is likely due to the fact that the indexed position in a zip file is not properly tracked. Unfortunately I don't have a good ETA on a fix for you. It is at least one major release away from being addressed. As a workaround, i suggest monitoring the files before they are zipped or setup a script to decompress the zip files into another directory that splunk can monitor.

TRAAAL3
Explorer

When we have a solution for this problem?

0 Karma
Get Updates on the Splunk Community!

Threat Hunting Unlocked: How to Uplevel Your Threat Hunting With the PEAK Framework ...

WATCH NOWAs AI starts tackling low level alerts, it's more critical than ever to uplevel your threat hunting ...

Splunk APM: New Product Features + Community Office Hours Recap!

Howdy Splunk Community! Over the past few months, we’ve had a lot going on in the world of Splunk Application ...

Index This | Forward, I’m heavy; backward, I’m not. What am I?

April 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...