When we receive this message or warning saying 100MB threshold is reached, the output what we see, do we say whatever it has given with the search executed, is that accurate data??
Or this data cannot be considered as accurate because of 100MB threshold??
All, I wanted to understand is even with this 100MB threshold, the results which is given do we say it is accurate or it is partial and cannot be considred as Accurate enough and it needs to be optimised or run by a search admin with high disk quota to get accurate results...
First,I think you need to optimise your search query
and secondly in search head $SPLUNK_HOME/etc/system/local/authorize.conf put
srchDiskQuota = 500
Maximum amount of disk space (MB) that can be used by search jobs of a user that belongs to this role which is 500 MB by default its 100 MB your_role is allowed to take up 500 megabytes total on disk for all their jobs.
The first thing you might consider doing is using the | fields command at the earliest point possible to eliminate everything but the fields you need. Other than that, you'd have to post a non-confidential version of the search to answers, in a separate question, and we could see how to help you make it more space-efficient.
You must be running a heavy search which , for it's processing, taking more than 100mb of dispatch directory and thus getting finalized. You should look at optimizing your search to reduce it's footprint (recommended) or adjust srchDiskQuota for your role in authorize.conf to increase the disk usage limit.