Splunk Enterprise

Send Splunk Archive Logs to Ceph S3

splunkuser109
Explorer

How can we automatically send frozen/archived splunk logs from the indexers over to a Ceph S3 bucket using the indexers.conf file on the indexers? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, sort of.  Use indexers.conf to specify a coldToFrozenScript.  That script, which you must write, will copy the archived buckets to Ceph.  See https://docs.splunk.com/Documentation/Splunk/8.1.3/Indexer/Automatearchiving and https://docs.splunk.com/Documentation/Splunk/8.1.3/Admin/Indexesconf#PER_INDEX_OPTIONS

---
If this reply helps you, Karma would be appreciated.
0 Karma

splunkuser109
Explorer
  • Hmm so it doesn’t look like there’s an easy way for us to automatically copy over the frozen logs directly to the ceph s3 bucket? 

Do you have any ideas on how we can write a script to copy over frozen buckets over to ceph s3 buckets? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Personally, I like Visual Studio Code, but notepad++ is good, too.  😀

---
If this reply helps you, Karma would be appreciated.
0 Karma

splunkuser109
Explorer

hahaha. Can remotePath (ceph s3 bucket) not be used to store the cold or frozen buckets/logs? 

 

https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Indexesconf

0 Karma

richgalloway
SplunkTrust
SplunkTrust

remotePath is used for warm/cold buckets.  This is part of the SmartStore feature.  Frozen buckets are different and are not stored by SmarStore.

There is an example coldToFrozenScript in $SPLUNK_HOME/bin.

---
If this reply helps you, Karma would be appreciated.
0 Karma
Get Updates on the Splunk Community!

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...