Logs retention policy

[SS1]

Hi SPlunkers,

We have multiple sources reporting to same index, what we observe is for few sources we can see the searchable earliest event from the month of September and for few sources the earliest event from August. Does different source have different retention policies? or the data is stored in different buckets which are searchable

Retention policy for the index:

Setting Value

maxTotalDataSizeMB

307200

frozenTimePeriodInSecs

188697600

homePath.maxDataSizeMB

0

coldPath.maxDataSizeMB

0

[somesoni2]

Have a look at very useful Splunk documentation about Splunk retention policy here: https://docs.splunk.com/Documentation/Splunk/8.0.6/Indexer/Setaretirementandarchivingpolicy#Freeze_d...

Total Index size does cause older data to be frozen, so that should be the issue here as retention period is quite large.

[SS1]

Thanks for the reply.

Could this happen because of the index size which is why the logs older than september are not searchable?  we are able to view fluentd logs in the same index

We currently have a 300GB index, which saves about 3 weeks given current ingestion rates. 

[somesoni2]

The retention policy applies at index level. All the sources/sourcetypes within that index will have same retention period.

Having said that, the retention period of N days means that data will be completely available for at least N days. Data older than N days may be seen but they're undergoing frozen process. The reason for partial data past retention period is how Splunk stores and freezes data. Data is stored in buckets and a bucket is only frozen when all events in that bucket are older than retention period. Till that happens data from that bucket will be searchable (hence partial data).

Your retention period is set to 188697600 (2184 days), so you should see all the data for this years Aug/Sep. It could happen that different sources started sending data to Splunk at different time, hence the oldest data you see have different dates for different sources.

[SS1]

Thanks for the reply.

Could this happen because of the index size which is why the logs older than september are not searchable?  we are able to view fluentd logs in the same index

We currently have a 300GB index, which saves about 3 weeks given current ingestion rates. 

Is there any other setting we need to look into it?