Getting Data In

indexes.conf - Can multiple indexers save to same "cold" directory?

zindain24
Path Finder

We have two indexers and a single search head:
Splunk1
Splunk2
Splunkweb

Both Splunk1 and Splunk2 have their own independant SAN drives for Hot and Warm storage. For Cold storage, Splunk1 and Splunk2 share a NAS drive with enough space for extended retention.

Can I have Splunk1 and Splunk2 write to the same "Cold" directory for each index? Or should they be different directories for each indexer?

For example - Same Index:

Splunk1 and Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb/colddb
coldPath = /n01/data/spl/databasedb/colddb

OR - Different Path:

Splunk1: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb1/colddb
coldPath = /n01/data/spl/databasedb1/colddb

Splunk2: $SPLUNK_HOME/etc/system/local/indexes.conf

coldPath = /n01/data/spl/internetdb2/colddb
coldPath = /n01/data/spl/databasedb2/colddb

Thanks

0 Karma
1 Solution

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

View solution in original post

yannK
Splunk Employee
Splunk Employee

No they have to use different folders for homePath and coldPath.

This is because :

  • first, each buckets need to have an unique ID per index. otherwise the indexer will detect a duplicate and stop until manually fixed.
  • second, having multiple splunkd process maintaining the retention/rotation rules the same folders will lead to locks and errors.

yannK
Splunk Employee
Splunk Employee

thank you for the addendum mister Sowings.

0 Karma

sowings
Splunk Employee
Splunk Employee

They could use different paths to the same storage partition. The issues Yann describe would arise if both indexers were attempting to use the same target directory as the location for their cold buckets.

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...