Knowledge Management

On-prem to Smartstore migration issues- Why are the event counts not matching?

bsrikanthreddy5
Path Finder

Hi, 

We are migrating our cluster from on-prem to a smart-store enabled cluster in AWS, a few indexes at a time, during which process event counts is not matching in some cases.

Case1: Eventcount in aws cluster is less than event count in on-prem

Case2: Eventcount in aws cluster is more than event count in on-prem


Any idea what might cause the event count, not to match? 

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

How are you migrating the data?

How are you counting the data in each location?  Are you sure the data has finished migrating before you count it?

---
If this reply helps you, an upvote would be appreciated.
0 Karma

bsrikanthreddy5
Path Finder

I have added below stanza on my on-prem indexes.conf and server.conf

indexes.conf

[volume:splunk]
storageType = remote
path = s3://<bucketName>
remote.s3.endpoint = https://s3.us-east-2.amazonaws.com
remote.s3.encryption = sse-kms
remote.s3.kms.key_id =
remote.s3.access_key =
remote.s3.secret_key =

Server.conf
[cachemanager]
max_concurrent_uploads = 2
eviction_policy = noevict
hotlist_recency_secs = 3888000

adding remotePath = volume:splunk/$_index_name specific index which I plan to migrate and rolling restart the indexer cluster. 

I am using below commands to compare  the event count .  
| tstats count where index=<IndexName> by index
| dbinspect index=<IndexName> | dedup bucketId |  stats sum(eventCount) by index

I am monitoring the migration status using  below command on CM.

/opt/splunk/bin/splunk search "|rest /services/admin/cacheman/_metrics |fields splunk_server migration*" -auth admin:$(cat /etc/splunk/password) . 

 

Tags (1)
0 Karma
Get Updates on the Splunk Community!

Improve Your Security Posture

Watch NowImprove Your Security PostureCustomers are at the center of everything we do at Splunk and security ...

Maximize the Value from Microsoft Defender with Splunk

 Watch NowJoin Splunk and Sens Consulting for this Security Edition Tech TalkWho should attend:  Security ...

This Week's Community Digest - Splunk Community Happenings [6.27.22]

Get the latest news and updates from the Splunk Community here! News From Splunk Answers ✍️ Splunk Answers is ...