Curious how this was caused? Is it because the app was removed in the deployer?
Error pulling configurations from the search head cluster captain (https://myhost:8089); consider performing a destructive configuration resync on this search head cluster member
04-01-2015 22:24:59.846 +0000 WARN ConfReplicationThread - Error pulling configurations from captain=https://myhost:8089, consecutiveErrors=16057: Error in fetchLookupTableContents, asset_id=229e28b611b458fe1c05b7ed1d8e83cb0f8b9481: Non-200 status_code=500: Application does not exist: sos
Need to confirm ... how about those apps... they are system related (6.2.3) :
What about if IU find these apps on the shcluster/apps at the deployer ???
If I remove them, they get removed on the Search Heads, right ?
To fix this, on all search heads : a manual backup
Then remove the folders from shbundle, deploy from deployer;
And then return backup on each search head ?
I have other odd error , but in the other way, from the Search Head to the captain :
splunkd.log:05-25-2015 21:13:18.267 -0300 WARN ConfReplicationThread - Error pushing configurations to captain=https://:8089, consecutiveErrors=1: Error in acceptPush: Non-200 status_code=400: Error parsing JSON response: String value too long
metrics.log:05-25-2015 21:18:56.132 -0300 INFO Metrics - group=captainstability, stablefollowerpct=100, stablecaptainpct=0, numpolledcaptain=0, numpolledfollower=155, numpolledcandidate=0, upgradestocaptain=0, downgradesfromcaptain=0, captainchanges=0
metrics.log:05-25-2015 21:18:56.133 -0300 INFO Metrics - group=subtaskcounts, name=shccaptainartifact, tofixrepfactor=0, tofixadded=0, tofixremoved=0, tofixtotal=0, count=0
metrics.log:05-25-2015 21:18:56.133 -0300 INFO Metrics - group=subtaskseconds, name=shccaptainservice, seconds=0.000