Hi @kiran_panchavat @richgalloway
thankyou for your response!
Full error message: Error message : [Indexer-x] The search process with search_id="remote_ip-<xxx>" may have returned partial results. Try running your search again. If you see this error repeatedly, review search.log for details or contact your Splunk administrator.
We implemented smartstore for a high volume index a few weeks ago and have since been experiencing issues with search and replication factors not meeting for smartstore enabled indexes. We raised a support case with Splunk about this and they informed us that it is a known bug with no fix version available. We hadn't seen this issue before last week, but it's been showing up in most searches for the past 3-4 days (with smartstore index).
Furthermore, we are experiencing search performance issues on the smartstore index; either it takes a long time to fetch results or the search gets stuck if we run the query for more than 24 hours.
index configuration:
[smartstore_index]
frozenTimePeriodInSecs = 15552000
repFactor = auto
maxDataSize = 1000
maxHotBuckets = 30
hotlist_recency_secs = 86400maxGlobalRawDataSizeMB = 62914560
homePath = /data/smartstore_index/db
coldPath = /data/smartstore_index/colddb
thawedPath = /data/smartstore_index/thaweddb
remotePath = volume:remote_store/smartstore_index
Approx daily ingestion on index: 2TB per day. Local SSD volume size: 16TB Remote location: S3 bucket
We're not sure, if we're receiving this error because of search & replication factor issue, Request help to fix.
... View more
