Deployment Architecture

Edit hot/warm/cold data retentions

MAShawky
Explorer

Hello
I want to add below configuration to specific indexer
Hot/Warm/Cold Data retention 6 months 1.75TB
Frozen Data retention 6 months

configuration is

[myindex]
coldPath = $path\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $path\db
maxTotalDataSizeMB = 1835008
thawedPath = $path\thaweddb
maxDataSize = 1835008
frozenTimePeriodInSecs = 15780000

but when am trying to add anew index i got error like below
The following issues were found with submitted configuration: stanza=myindexparameter=maxDataSize Value supplied='1835008' is illegal; default='750'

0 Karma

hettervik
Builder

Hi,

From the doc on indexes.conf

maxDataSize = <positive integer>|auto|auto_high_volume
* The maximum size in MB for a hot DB to reach before a roll to warm is
  triggered.
* Specifying "auto" or "auto_high_volume" will cause Splunk to autotune this
  parameter (recommended).
* You should use "auto_high_volume" for high-volume indexes (such as the
  main index); otherwise, use "auto".  A "high volume index" would typically
  be considered one that gets over 10GB of data per day.
* Defaults to "auto", which sets the size to 750MB.
* "auto_high_volume" sets the size to 10GB on 64-bit, and 1GB on 32-bit
  systems.
* Although the maximum value you can set this is 1048576 MB, which
  corresponds to 1 TB, a reasonable number ranges anywhere from 100 to
  50000.  Before proceeding with any higher value, please seek approval of
  Splunk Support.*
* If you specify an invalid number or string, maxDataSize will be auto
  tuned.
* NOTE: The maximum size of your warm buckets may slightly exceed
  'maxDataSize', due to post-processing and timing issues with the rolling
  policy.

Note the max size on maxDataSize is 1048576. maxTotalDataSizeMB and frozenTimePeriodInSecs are the parameters you need to control the data size of your index. See more on archiving data (rolling to frozen) here:

http://docs.splunk.com/Documentation/Splunk/7.0.0/Indexer/Automatearchiving

0 Karma
Career Survey
First 500 qualified respondents will receive a $20 gift card! Tell us about your professional Splunk journey.

Can’t make it to .conf25? Join us online!

Get Updates on the Splunk Community!

Take Action Automatically on Splunk Alerts with Red Hat Ansible Automation Platform

 Are you ready to revolutionize your IT operations? As digital transformation accelerates, the demand for ...

Calling All Security Pros: Ready to Race Through Boston?

Hey Splunkers, .conf25 is heading to Boston and we’re kicking things off with something bold, competitive, and ...

Beyond Detection: How Splunk and Cisco Integrated Security Platforms Transform ...

Financial services organizations face an impossible equation: maintain 99.9% uptime for mission-critical ...