What process or search is associated with _events.stash_new?

[RNB]

The last few days I have been coming into work and the Splunk server is out of disk space. The culprit is always a 23+GB $SPLUNK/var/spool/splunk/___events.stash___new file that has filled the disk.

I cannot find a saved search with a last run date/time that corresponds with the creation time of the file.

How do I find the query or process that is associated with this stash file?

[LukeMurphey · Accepted solution]

Stash_new files are created when events are sent for summary indexing. Basically, when an event is sent for summary indexing, a stash file is created which is then read by Splunk which puts the contents into the appropriate index.

You can determine where the events are coming from by examining the contents of the stash file. The stash file will look something like:

***SPLUNK*** index=summary source="Some search"
==##~~##~~  1E8N3D4E6V5E7N2T9 ~~##~~##==
05/13/2011 14:35:00, src=whatever dst=etc <...event goes here>
==##~~##~~  1E8N3D4E6V5E7N2T9 ~~##~~##==

Note that you can determine the source (the search that created the event) and the index that the events are being written to by looking at the first line (just after "*********SPLUNK*********").