Splunk Cloud Platform

Can I archive data in Splunk Cloud?

adukes_splunk
Splunk Employee
Splunk Employee

Since I don't have an on-premise storage option, how can I keep data beyond my 90 day retention allocation?

0 Karma
1 Solution

adukes_splunk
Splunk Employee
Splunk Employee

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

How indexes help you manage, store, and restore data

Indexes store the data sent to your Splunk Cloud deployment. You can create, update, delete, and view index properties, modify data retention settings for individual indexes, delete data from indexes, and optimize search performance by managing the number of indexes and the data sources stored in specific indexes. See manage Splunk Cloud indexes to learn best practices for indexes.

The Splunk Cloud service bases your storage space on the volume of uncompressed data you want to index daily and comes with enough storage to store up to 90 days of uncompressed data. For example, if your daily volume of uncompressed data is 100 GB, your Splunk Cloud has 9000 GB (9 TB) of storage. When the index reaches the specified days retention, Splunk Cloud deletes the oldest data from the index.

If you need to store data beyond your retention allocation and have a Managed Splunk Cloud, you can augment Splunk Cloud with Dynamic Data Self Storage (DDSS) or Dynamic Data Active Archive (DDAA). DDSS is available by default to Splunk Cloud customers. DDAA is a low-cost option to move your data to a Splunk-maintained searchable archive.

Splunk Cloud places the data you send in indexes you self-manage from the Indexes page in Splunk Web. Splunk Cloud retains data based on index settings that enable you to specify when to delete data. Review the Splunk Cloud data policies before you configure data retention settings for different data sources. Data is not searchable after Splunk Cloud deletes it from the index. It's a best practice to store data in separate indexes to meet your audit and compliance requirements.

Things to know

If you've configured DDAA or DDSS as data ages from searchable old, data automatically moves to the appropriate repository when the storage meets the retention setting for an index. The Splunk Cloud Monitoring (CMC) app is part of Splunk Cloud and is available to help you monitor Splunk Cloud deployment health. CMC displays details about your storage consumption and details such as data stored, number of days of retention for each index.

DDSS: Exports your oldest data to your AWS S3 account before deleting it from the index. Review the requirements for Dynamic Data Self Storage to see how to export of your aged, ingested data. Also see Configure self storage locations on Amazon S3 and Dynamic Data: Self-Storage – Compliance, Cloud and Data Lifecycle.

DDAA: Stores data until the retention setting that you specify expires. Your DDAA subscription entitles you to restore up to 10% of your archive subscription per restore. Restored data is searchable within 24 hours of the restore time and remains searchable for up to 30 days. If multiple restores overlap within a 30-day period, it accrues against the restore entitlement. See Dynamic Data: Data Retention Options in Splunk Cloud.

Things to do

View solution in original post

adukes_splunk
Splunk Employee
Splunk Employee

The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.

How indexes help you manage, store, and restore data

Indexes store the data sent to your Splunk Cloud deployment. You can create, update, delete, and view index properties, modify data retention settings for individual indexes, delete data from indexes, and optimize search performance by managing the number of indexes and the data sources stored in specific indexes. See manage Splunk Cloud indexes to learn best practices for indexes.

The Splunk Cloud service bases your storage space on the volume of uncompressed data you want to index daily and comes with enough storage to store up to 90 days of uncompressed data. For example, if your daily volume of uncompressed data is 100 GB, your Splunk Cloud has 9000 GB (9 TB) of storage. When the index reaches the specified days retention, Splunk Cloud deletes the oldest data from the index.

If you need to store data beyond your retention allocation and have a Managed Splunk Cloud, you can augment Splunk Cloud with Dynamic Data Self Storage (DDSS) or Dynamic Data Active Archive (DDAA). DDSS is available by default to Splunk Cloud customers. DDAA is a low-cost option to move your data to a Splunk-maintained searchable archive.

Splunk Cloud places the data you send in indexes you self-manage from the Indexes page in Splunk Web. Splunk Cloud retains data based on index settings that enable you to specify when to delete data. Review the Splunk Cloud data policies before you configure data retention settings for different data sources. Data is not searchable after Splunk Cloud deletes it from the index. It's a best practice to store data in separate indexes to meet your audit and compliance requirements.

Things to know

If you've configured DDAA or DDSS as data ages from searchable old, data automatically moves to the appropriate repository when the storage meets the retention setting for an index. The Splunk Cloud Monitoring (CMC) app is part of Splunk Cloud and is available to help you monitor Splunk Cloud deployment health. CMC displays details about your storage consumption and details such as data stored, number of days of retention for each index.

DDSS: Exports your oldest data to your AWS S3 account before deleting it from the index. Review the requirements for Dynamic Data Self Storage to see how to export of your aged, ingested data. Also see Configure self storage locations on Amazon S3 and Dynamic Data: Self-Storage – Compliance, Cloud and Data Lifecycle.

DDAA: Stores data until the retention setting that you specify expires. Your DDAA subscription entitles you to restore up to 10% of your archive subscription per restore. Restored data is searchable within 24 hours of the restore time and remains searchable for up to 30 days. If multiple restores overlap within a 30-day period, it accrues against the restore entitlement. See Dynamic Data: Data Retention Options in Splunk Cloud.

Things to do

Get Updates on the Splunk Community!

Federated Search for Amazon S3 | Key Use Cases to Streamline Compliance Workflows

Modern business operations are supported by data compliance. As regulations evolve, organizations must ...

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...