Getting Data In

indexes.conf - Can multiple indexers save to same "cold" directory?

zindain24
Path Finder

We have two indexers and a single search head:
Splunk1
Splunk2
Splunkweb

Both Splunk1 and Splunk2 have their own independant SAN drives for Hot and Warm storage. For Cold storage, Splunk1 and Splunk2 share a NAS drive with enough space for extended retention.

Can I have Splunk1 and Splunk2 write to the same "Cold" directory for each index? Or should they be different directories for each indexer?

For example - Same Index:

Splunk1 and Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb/colddb
coldPath = /n01/data/spl/databasedb/colddb

OR - Different Path:

Splunk1: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb1/colddb
coldPath = /n01/data/spl/databasedb1/colddb

Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb2/colddb
coldPath = /n01/data/spl/databasedb2/colddb

Thanks

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

View solution in original post

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

yannK
Splunk Employee
Splunk Employee

thank you for the addendum mister Sowings.

0 Karma

sowings
Splunk Employee
Splunk Employee

They could use different paths to the same storage partition. The issues Yann describe would arise if both indexers were attempting to use the same target directory as the location for their cold buckets.

0 Karma
Get Updates on the Splunk Community!

Observe and Secure All Apps with Splunk

  Join Us for Our Next Tech Talk: Observe and Secure All Apps with SplunkAs organizations continue to innovate ...

Splunk Decoded: Business Transactions vs Business IQ

It’s the morning of Black Friday, and your e-commerce site is handling 10x normal traffic. Orders are flowing, ...

Fastest way to demo Observability

I’ve been having a lot of fun learning about Kubernetes and Observability. I set myself an interesting ...