Lost AWS events after ingestion

[vishalduttauk]

I am in the middle of a Splunk migration. One of the tasks is to moved data from some sourcetypes onto the new servers using the | collect index=aws sourcetype=* command.

The numbers added up after running checks. I run the same checks again a day later and the numbers no longer match up.

Source 1 ->	Old Splunk	New Splunk	Source 2 ->	Old Splunk	New Splunk
August	12,478,853	12,478,853		26,171,911	26,171,911

24 hours later

Source 1 ->	Old Splunk	New Splunk	Source 2 ->	Old Splunk	New Splunk
	12,478,853	12,477,696		26,171,911	3,001,183

I've set the following stanza within the indexes.conf file on the deployment server. Also the index only contains 22gb of data. Can you help?

[aws]

coldPath = $SPLUNK_DB\$_index_name\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB\$_index_name\db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB\$_index_name\thaweddb
frozenTimePeriodInSecs=94608000

[PickleRick]

Most probably your data was rolled out due to either retention period or index/volume size limits.

[tej57]

Hello @vishalduttauk,

Can you provide the complete search that you used for migrating the data from one server to other?

Thanks,
Tejas. 

[vishalduttauk]

index=aws but i ended up logging onto both servers and moving the whole index from "old" Splunk over to "new" Splunk