Deployment Architecture

When the hot bucket is created on clustered envioemnet. What steps are followed.

sat94541
Communicator

When the hot bucket is created on clustered envioemnet. What steps are followed.

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

When an indexer creates a hot bucket, it follows this procedure:

1) Indexer needs to create a new hot bucket
2) Indexer asks the CM whom to replicate the new hot bucket to
3) CM receives the request, checks the configured RF/SF, and selects indexers (randomly) as "targets" that the original indexer should stream to. For example, if RF=3, SF=2, it will respond with two targets, one of which will also be searchable (to satisfy SF=2).

All hot bucket replications are raw data ONLY. For the example above, the one target that is also told to be searchable will create its own tsidx files based on the rawdata that comes in.

rbal_splunk
Splunk Employee
Splunk Employee

Yes, we can survive failures, i.e. we won't stop indexing, but we cannot possibly meet replication policy for those source indexers that have a now failed peer in their target list. CM recovery and fixup needed asap, in that case
In other words: We really need CM HA.

0 Karma

sat94541
Communicator

Does splunk say that ..... if your CM is down....we can survive any additional failures

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

correct.in which case new buckets dont meet rf/sf

0 Karma

sat94541
Communicator

Until one of them fails as well, in which case.....?
The cluster cannot remain healthy until the CM comes back up...?

0 Karma

sat94541
Communicator

Until one of them fails as well, in which case.....?
The cluster cannot remain healthy until the CM comes back up...?

0 Karma

sat94541
Communicator

so with RF 3, it's going to be the same two indexers over and over

0 Karma

rbal_splunk
Splunk Employee
Splunk Employee

each indexer remembers the LAST list of targets the CM gave it. if the CM is down, it will continuously use the same targets for new hot buckets.
to be more correct, each indexer remembers the "last response of a new hot bucket request", and reuses that response

0 Karma

sat94541
Communicator

we've believe that if the CM is down, we don't have to really break our necks in bringing it back up, e.g. an hour or more will be OK; in this instance, does the indexer always use the same peers to replicate to, or does it have a list of them it can use?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...