Using Splunk ES 5.3.1, I have a saved search that reached the 25GB limit (srchDiskQuota) before being finalized. This ran two days in a row and ended up filling my dispatch directory. In total it was searching over 65 billion events over the 30 day time period in the Web datamodel.
Looking through the jobs I was able to identify the search and disabled it from running further. However, I don't know where this search is used in ES and where the results are used. I'd like to determine that so I know what will be missing and where by disabling this search. The only information I have found is that it is used in the Machine Learning Tool Kit but I don't have MLTK installed in ES nor is it an applicable version.
Name: Web - Web Event Count By Src By HTTP Method Per 1d - Context Gen
App: SA-NetworkProtection
Type: saved search
Location: /opt/splunk/etc/apps/SA-NetworkProtection/default/savedsearches.conf
[Web - Web Event Count By Src By HTTP Method Per 1d - Context Gen]
action.email.sendresults = 0
cron_schedule = 0 0 * * *
disabled = False
dispatch.earliest_time = -31d@d
dispatch.latest_time = -1d@d
enableSched = 1
is_visible = false
schedule_window = 20
search = | tstats `summariesonly` count as web_event_count from datamodel=Web.Web by Web.src, Web.http_method, _time span=24h | `drop_dm_object_name("Web")` | where match(http_method, "^[A-Za-z]+$") | `context_stats(web_event_count, http_method)` | eval min=0 | eval max=median*2 | xscreateddcontext name=count_by_http_method_by_src_1d container=web class=http_method app="SA-NetworkProtection" scope=app type=domain terms=`xs_default_magnitude_concepts` | stats count
