Splunk Enterprise

Send Splunk Archive Logs to Ceph S3

splunkuser109
Explorer

How can we automatically send frozen/archived splunk logs from the indexers over to a Ceph S3 bucket using the indexers.conf file on the indexers? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Yes, sort of.  Use indexers.conf to specify a coldToFrozenScript.  That script, which you must write, will copy the archived buckets to Ceph.  See https://docs.splunk.com/Documentation/Splunk/8.1.3/Indexer/Automatearchiving and https://docs.splunk.com/Documentation/Splunk/8.1.3/Admin/Indexesconf#PER_INDEX_OPTIONS

---
If this reply helps you, an upvote would be appreciated.
0 Karma

splunkuser109
Explorer
  • Hmm so it doesn’t look like there’s an easy way for us to automatically copy over the frozen logs directly to the ceph s3 bucket? 

Do you have any ideas on how we can write a script to copy over frozen buckets over to ceph s3 buckets? 

0 Karma

richgalloway
SplunkTrust
SplunkTrust

Personally, I like Visual Studio Code, but notepad++ is good, too.  😀

---
If this reply helps you, an upvote would be appreciated.
0 Karma

splunkuser109
Explorer

hahaha. Can remotePath (ceph s3 bucket) not be used to store the cold or frozen buckets/logs? 

 

https://docs.splunk.com/Documentation/Splunk/8.1.0/Admin/Indexesconf

0 Karma

richgalloway
SplunkTrust
SplunkTrust

remotePath is used for warm/cold buckets.  This is part of the SmartStore feature.  Frozen buckets are different and are not stored by SmarStore.

There is an example coldToFrozenScript in $SPLUNK_HOME/bin.

---
If this reply helps you, an upvote would be appreciated.
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...