Running 5.0.1 on Linux, receiving this error over 500 times a day spread across 34 indexers. Using the splunk service account, I was able to untar the the bundle fine so I do not believe it's permission based. Any suggestions on how to resolve the error? I've posted more warnings around the error below:
splunkd.log.5:02-10-2013 19:40:16.107 +0000 ERROR DistBundleRestHandler - Problem untarring file: /opt/splunk/var/run/searchpeers/82C8F892-6A60-451B-9E7B-012BC192118F-1360524901.bundle
splunkd.log.5:02-10-2013 20:06:22.449 +0000 WARN DistBundleRestHandler - There was a problem renaming: /opt/splunk/var/run/searchpeers/82C8F892-6A60-451B-9E7B-012BC192118F-1360526701.tmp -> /opt/splunk/var/run/searchpeers/82C8F892-6A60-451B-9E7B-012BC192118F-1360526701: Directory not empty
splunkd.log.5:02-10-2013 20:06:22.449 +0000 ERROR DistBundleRestHandler - Problem untarring file: /opt/splunk/var/run/searchpeers/82C8F892-6A60-451B-9E7B-012BC192118F-1360526701.bundle
splunkd.log.5:02-10-2013 20:30:14.185 +0000 WARN DistBundleRestHandler - Removed pre-existing temporary directory for untar: /opt/splunk/var/run/searchpeers/DCC02B2A-C40A-41C1-BD92-434A555088B9-1360352701.tmp
splunkd.log.5:02-10-2013 20:30:14.505 +0000 WARN DistBundleRestHandler - There was a problem renaming: /opt/splunk/var/run/searchpeers/DCC02B2A-C40A-41C1-BD92-434A555088B9-1360352701.tmp -> /opt/splunk/var/run/searchpeers/DCC02B2A-C40A-41C1-BD92-434A555088B9-1360352701: Directory not empty
splunkd.log.5:02-10-2013 20:30:14.505 +0000 ERROR DistBundleRestHandler - Problem untarring file: /opt/splunk/var/run/searchpeers/DCC02B2A-C40A-41C1-BD92-434A555088B9-1360352701.bundle
Bundle replication failures can happen for a variety of reasons. That being said, the ERROR messages here implicate SPL-60740/SPL-74416 as a possible cause -- Splunk instances in a search head pool, under certain conditions, attempt to perform replications that have already completed in the past. This then causes "collisions" with the pre-existing bundles on the indexer.
The fix for this bug will land in future 5.0.x and 6.0.x maintenance releases.
I'm having the same problem between my deployment server and index cluster. If I remove all the files and directories starting with my deployment server's name from $SPLUNK_HOME/var/run/searchpeers it recovers with no need to restart,
Splunk 7.0.3 (build fa31da744b51).
I am facing the same issue on 6.3.2 😞
Any work around?
encountered these same error messages "Problem untarring file: " and "Directory not empty" in a 6.1.3 release:
SPLUNK VERSION:
VERSION=6.1.3
BUILD=220630
PRODUCT=splunk
PLATFORM=Linux-x86_64
I too am seeing this issue, however I don't see this on the known issues list in 5.0.4
Bundle replication failures can happen for a variety of reasons. That being said, the ERROR messages here implicate SPL-60740/SPL-74416 as a possible cause -- Splunk instances in a search head pool, under certain conditions, attempt to perform replications that have already completed in the past. This then causes "collisions" with the pre-existing bundles on the indexer.
The fix for this bug will land in future 5.0.x and 6.0.x maintenance releases.
This is a verified bug SPL-62238. The fix will be resolved in v5.0.4.
mmmm...
Waiting mode: on
:)
The fix is going to land in future 5.0.x and 6.0.x maintenance releases.
I'm seeing this same issue on 6.0. Did you ever resolve it?
The fix is going to land in a future 5.0.x maintenance release. It didn't really land in 5.0.4.
Ping! Any updates?
I'm seeing still seeing the same issue with 5.0.4 as well. Any updates?
I'm experiencing the same issue.
I did upgrade to version 5.0.4 but the problem persists...
How to check status of bug SPL-62238?
when can we expect to be able to download 5.0.4 ? 🙂
Did you ever figure out the solution to this issue? I am experiencing the same issues myself.