Splunk Search

Why cant I see some data that I was able to see before 1 month? Even if retention policy of index is 3 years

muez
Explorer

Notes
- Our retention policy is 3 years for that abc index.
- When I exported the result of that query before 1 month, I was able to see that particular data
- Today when I run exact same query, I can see some missing data.
- To give you the detail, today I am seeing approx 20K less events out of 1L events.
- The date range is exact same

0 Karma

darrenfuller
Contributor

On your indexing layer, run the following from the command line:

splunk btool indexes list <INDEXNAME> --debug

Replacing with the name of the index that you are seeing issues with. There are a few properties to take note of:

1) coldPath.maxDataSizeMB -- The total size in MB of the Cold path for data. If this size is exceeded, data will roll to frozen (and if there is no Cold-To-Frozen archiving strategy in place, will be deleted)
2) frozenTimePeriodInSecs -- The number of seconds before data is frozen
3) maxTotalDataSizeMB -- The maximum total size across all hot/warm/cold data locations

See if any of these are lower than you expect.

0 Karma

manjunathmeti
Champion

Check if data is deleted because of retention or max size in last 1 month.

index=_internal sourcetype=splunkd INDEX_NAME component=BucketMover frozenTimePeriodInSecs OR maxTotalDataSizeMB
0 Karma

muizash
Path Finder

@manjunathmeti The above query is not running, and also data data cant be deleted because retention is 3 years and time stamp of data was in january 2020 only

0 Karma
Get Updates on the Splunk Community!

.conf24 | Registration Open!

Hello, hello! I come bearing good news: Registration for .conf24 is now open!   conf is Splunk’s rad annual ...

ICYMI - Check out the latest releases of Splunk Edge Processor

Splunk is pleased to announce the latest enhancements to Splunk Edge Processor.  HEC Receiver authorization ...

Introducing the 2024 SplunkTrust!

Hello, Splunk Community! We are beyond thrilled to announce our newest group of SplunkTrust members!  The ...