Splunk Enterprise Security

Incomplete search reults error

sahityasweety
Explorer

I am getting this error,

may have returned partial results try running your search again.if you see this error repeatedly, review search.log for details or contact your Splunk administration

Can i please get solution for this

 

 

Thanks,

sahitya

Labels (1)
0 Karma

kiran_panchavat
Contributor

@sahityasweety 

Check the Job Inspector. In the Inspector popup window, there is another link to the search.log that gives you some very detailed information. Another way to see more info about your errors is to open a plain search window and do a search like:

index=_internal error

Review your search query to ensure it is correctly formulated. Check for any typos, missing keywords, or syntax errors.

Verify that the time range specified in your search covers the relevant data.

Ensure that you are searching within the appropriate indexes.

Check the Splunk logs (such as splunkd.log and search.log) for any errors or warnings related to your search.

https://lantern.splunk.com/Splunk_Platform/Product_Tips/Searching_and_Reporting/Troubleshooting_and_...


https://docs.splunk.com/Documentation/SplunkCloud/9.1.2312/Search/ViewsearchjobpropertieswiththeJobI...r

If this reply helps you, Karma would be appreciated.

0 Karma

sahityasweety
Explorer

Hi @kiran_panchavat  @richgalloway 

thankyou for your response!

Full error message: Error message : [Indexer-x] The search process with search_id="remote_ip-<xxx>" may have returned partial results. Try running your search again. If you see this error repeatedly, review search.log for details or contact your Splunk administrator.

We implemented smartstore for a high volume index a few weeks ago and have since been experiencing issues with search and replication factors not meeting for smartstore enabled indexes. We raised a support case with Splunk about this and they informed us that it is a known bug with no fix version available. We hadn't seen this issue before last week, but it's been showing up in most searches for the past 3-4 days (with smartstore index).

Furthermore, we are experiencing search performance issues on the smartstore index; either it takes a long time to fetch results or the search gets stuck if we run the query for more than 24 hours.

index configuration:
[smartstore_index]
frozenTimePeriodInSecs = 15552000
repFactor = auto
maxDataSize = 1000
maxHotBuckets = 30
hotlist_recency_secs = 86400maxGlobalRawDataSizeMB = 62914560
homePath = /data/smartstore_index/db
coldPath = /data/smartstore_index/colddb
thawedPath = /data/smartstore_index/thaweddb
remotePath = volume:remote_store/smartstore_index

Approx daily ingestion on index: 2TB per day.
Local SSD volume size: 16TB
Remote location: S3 bucket

We're not sure, if we're receiving this error because of search & replication factor issue, Request help to fix.

0 Karma

richgalloway
SplunkTrust
SplunkTrust

That error usually is seen when an indexer stops while processing a search.  What relevant messages do you seen in search.log?  Are there any core files from the indexers?

The local SSD provides only 8 days of storage.  Any query that searches data more than 8 days old will have to fetch the data from SmartStore, which will slow processing of the search.  Splunk recommends the local cache be large enough to hold at least 30 days of data.

---
If this reply helps you, Karma would be appreciated.

richgalloway
SplunkTrust
SplunkTrust

Did you run the search again and get the same message?  If so, what did you find in search.log?

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

New Dates, New City: Save the Date for .conf25!

Wake up, babe! New .conf25 dates AND location just dropped!! That's right, this year, .conf25 is taking place ...

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...