Splunk Enterprise Security

Bucket Flap

adol83
Explorer

Hello,
I'm new here and I wanted some help for this issue.
My incident is getting many errors for a bucket replication that keeps flapping up/down. In the master dashboard I have the errors "search factor is not met" and "replication factor is not met" along with main page warnings like "msg='target doesn't have bucket now. ignoring' " and "making bucket serviceable, we have enough peers now " that suggests me it's flapping other than the up/down I see in the master dashboard.

I have a little infrastructure with

1 Master
2 Indexers
1 Search Head
1 Heavy Forwarder

My configuration on local (that should override the default server.conf) is fine having replication_factor=2 and search_factor=2 but it seems that no matter which change I apply the always stays up.
I tried to resync the bucket but actually I'm not even sure it did it. However, among my fix up tasks I have 2, 1 for replication factor and 1 for search factor

For what concern search factor I have the following:

fixup reason: unmet rf
current status: Missing enough suitable candidates to create searchable copy in order to meet replication policy. Missing={ default:1 }

for what concern replication factor:

fixup reason: unmet rf
current status: empty

could you please let me know?

I have some basic knowledge of administration and clustering by reading Splunk docs but I'm not sure I am really into yet.
splunk btool server list --debug
give me an output whereas replication_factor in local config is 2 and in default config is 3 but as far as I know local config in this case should override the default one.

I'm stuck!

Thank you in advance

1 Solution

codebuilder
SplunkTrust
SplunkTrust

Your issue is with the search_factor setting. It cannot be set to a value of 2 with a single search head. Search artifacts are stored on the search heads. Since you have only one search head, but a setting of 2, it is trying to replicate artifacts but nowhere to put them.

----
An upvote would be appreciated and Accept Solution if it helps!

View solution in original post

adol83
Explorer

Thank you: I will apply your fix but your explanation has already been pretty straightforwarded.

0 Karma

codebuilder
SplunkTrust
SplunkTrust

Your issue is with the search_factor setting. It cannot be set to a value of 2 with a single search head. Search artifacts are stored on the search heads. Since you have only one search head, but a setting of 2, it is trying to replicate artifacts but nowhere to put them.

----
An upvote would be appreciated and Accept Solution if it helps!
Get Updates on the Splunk Community!

Routing logs with Splunk OTel Collector for Kubernetes

The Splunk Distribution of the OpenTelemetry (OTel) Collector is a product that provides a way to ingest ...

Welcome to the Splunk Community!

(view in My Videos) We're so glad you're here! The Splunk Community is place to connect, learn, give back, and ...

Tech Talk | Elevating Digital Service Excellence: The Synergy of Splunk RUM & APM

Elevating Digital Service Excellence: The Synergy of Real User Monitoring and Application Performance ...