Splunk Search

Why cant I see some data that I was able to see before 1 month? Even if retention policy of index is 3 years

muez
Explorer

Notes
- Our retention policy is 3 years for that abc index.
- When I exported the result of that query before 1 month, I was able to see that particular data
- Today when I run exact same query, I can see some missing data.
- To give you the detail, today I am seeing approx 20K less events out of 1L events.
- The date range is exact same

0 Karma

darrenfuller
Contributor

On your indexing layer, run the following from the command line:

splunk btool indexes list <INDEXNAME> --debug

Replacing with the name of the index that you are seeing issues with. There are a few properties to take note of:

1) coldPath.maxDataSizeMB -- The total size in MB of the Cold path for data. If this size is exceeded, data will roll to frozen (and if there is no Cold-To-Frozen archiving strategy in place, will be deleted)
2) frozenTimePeriodInSecs -- The number of seconds before data is frozen
3) maxTotalDataSizeMB -- The maximum total size across all hot/warm/cold data locations

See if any of these are lower than you expect.

0 Karma

manjunathmeti
SplunkTrust
SplunkTrust

Check if data is deleted because of retention or max size in last 1 month.

index=_internal sourcetype=splunkd INDEX_NAME component=BucketMover frozenTimePeriodInSecs OR maxTotalDataSizeMB
0 Karma

muizash
Path Finder

@manjunathmeti The above query is not running, and also data data cant be deleted because retention is 3 years and time stamp of data was in january 2020 only

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...