Getting Data In

Lost AWS events after ingestion

vishalduttauk
Communicator

I am in the middle of a Splunk migration. One of the tasks is to moved data from some sourcetypes onto the new servers using the | collect index=aws sourcetype=* command.

The numbers added up after running checks. I run the same checks again a day later and the numbers no longer match up.

Source 1 -> Old Splunk New Splunk Source 2 -> Old Splunk New Splunk
August 12,478,853 12,478,853   26,171,911 26,171,911

 

24 hours later

Source 1 -> Old Splunk New Splunk Source 2 -> Old Splunk New Splunk
  12,478,853 12,477,696   26,171,911 3,001,183

 

I've set the following stanza within the indexes.conf file on the deployment server. Also the index only contains 22gb of data. Can you help?

[aws]

coldPath = $SPLUNK_DB\$_index_name\colddb
enableDataIntegrityControl = 0
enableTsidxReduction = 0
homePath = $SPLUNK_DB\$_index_name\db
maxTotalDataSizeMB = 512000
thawedPath = $SPLUNK_DB\$_index_name\thaweddb
frozenTimePeriodInSecs=94608000
Labels (2)
0 Karma
Get Updates on the Splunk Community!

Continuing Innovation & New Integrations Unlock Full Stack Observability For Your ...

You’ve probably heard the latest about AppDynamics joining the Splunk Observability portfolio, deepening our ...

Monitoring Amazon Elastic Kubernetes Service (EKS)

As we’ve seen, integrating Kubernetes environments with Splunk Observability Cloud is a quick and easy way to ...

Cloud Platform & Enterprise: Classic Dashboard Export Feature Deprecation

As of Splunk Cloud Platform 9.3.2408 and Splunk Enterprise 9.4, classic dashboard export features are now ...