Splunk Search

Why cant I see some data that I was able to see before 1 month? Even if retention policy of index is 3 years

Explorer

Notes
- Our retention policy is 3 years for that abc index.
- When I exported the result of that query before 1 month, I was able to see that particular data
- Today when I run exact same query, I can see some missing data.
- To give you the detail, today I am seeing approx 20K less events out of 1L events.
- The date range is exact same

0 Karma

Contributor

On your indexing layer, run the following from the command line:

splunk btool indexes list <INDEXNAME> --debug

Replacing with the name of the index that you are seeing issues with. There are a few properties to take note of:

1) coldPath.maxDataSizeMB -- The total size in MB of the Cold path for data. If this size is exceeded, data will roll to frozen (and if there is no Cold-To-Frozen archiving strategy in place, will be deleted)
2) frozenTimePeriodInSecs -- The number of seconds before data is frozen
3) maxTotalDataSizeMB -- The maximum total size across all hot/warm/cold data locations

See if any of these are lower than you expect.

0 Karma

Influencer

Check if data is deleted because of retention or max size in last 1 month.

index=_internal sourcetype=splunkd INDEX_NAME component=BucketMover frozenTimePeriodInSecs OR maxTotalDataSizeMB
0 Karma

Path Finder

@manjunathmeti The above query is not running, and also data data cant be deleted because retention is 3 years and time stamp of data was in january 2020 only

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!