Hello,
I wonder if you have any suggestion as to why, over time, results of a stats count may vary for a past time frame.
I have a planned report doint this search each week:
index=x1 OR index=x2 OR index=x3 OR index=x4
| eval tempo = strftime(_time,"%Y-%m")
| stats count by tempo,index
| sort by tempo, index
2 of the 4 indexes are closed (no new event since at least a year) => older events
Recent events are indexed on the other 2 indexes.
Over the past period in the report, I should have each week the same result (ie for 2016-04 index=x1 result_stats_count=125469522).
Except not.
Beginning 3 weeks ago, results changed over some period (2016-04 for example) even if there is no new data for that period of time (I checked : no new event has been indexed for these index this year)
In some cases, the number increases, in other it decreases, or both over 3 weeks. This data has not yet reach the retention limit.
The splunk platform is 2 SHC, 1 indexer cluster multisite and a few forwarders.
Operation that has been done the last 3 weeks : new cluster bundle conf with rolling restart, some SHC rolling restart.
I didn't find anyting helpfull in _internal explain this behaviour.
Do you have any idea ? Pointers ?
Thanks a lot,
Ema
