Splunk Search

Extremely large search job size

Alan_Chan
Explorer

We found that the search job size becomes extremely large during searches. My Splunk instance is a newly installed testing lab with the following limits.conf only.

Anyone else have any idea on this situation?

/opt/splunk/etc/system/local/limits.conf

[search]

read_final_results_from_timeliner = 1

Update:
When I change the read_final_results_from_timeliner = 1 to read_final_results_from_timeliner = true
The Job size reduced and I don't know why.
Alan_Chan_0-1746607565626.png

 

 

Labels (1)
Tags (1)
0 Karma

livehybrid
SplunkTrust
SplunkTrust

Hi @Alan_Chan 

The issue of an extremely large search job size can be caused by several factors. To troubleshoot, you should first check the search query that's causing the large job size.

| rest /services/search/jobs 
| search dispatchState="DONE" AND isFinalized=0
| sort - runDuration
| table sid, label, runDuration, scanCount, resultCount, diskUsage
| rename label as "Search Query"

This SPL will list the recent search jobs, sorted by their run duration, and provide details such as the search query, scan count, result count, and disk usage.

The limits.conf you've provided only contains one setting: read_final_results_from_timeliner = 1. This setting is related to how Splunk reads final results, but it doesn't directly explain the large search job size.

To mitigate large search job sizes, consider optimising your search queries to reduce the amount of data being processed and returned.

Use | stats or other transforming commands early in your search to reduce data volume.

Limit the time range of your searches. - Avoid using * or overly broad field names in your searches.

🌟 Did this answer help you? If so, please consider:

  • Adding karma to show it was useful
  • Marking it as the solution if it resolved your issue
  • Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...