https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84569#M21554 <P>We recently upgraded to Splunk 6 and on multiple occasions a real-time search seems to magically appear and causes all other searches/dashboards to halt because the limit for searches has been reached. When I check all running jobs I see 100s of real-time searches "<CODE>|</CODE>" (a single pipe) with no start-time or end-time by the user Admin. Initially, I thought this may be related to work on a real-time dashboard and post-process searches, but the dashboard is in a different app than where these mysterious searches are reportedly running.</P> <P>To resolve the issue, I restart Splunk and delete the searches out of the dispatch directory. Simply trying to stop/delete them from the job management app will not work. </P> <P>Is there any way to determine what is causing this? Could this be a bug in version 6?</P> Sun, 06 Oct 2013 15:30:23 GMT sc0tt 2013-10-06T15:30:23Z https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84569#M21554 <P>We recently upgraded to Splunk 6 and on multiple occasions a real-time search seems to magically appear and causes all other searches/dashboards to halt because the limit for searches has been reached. When I check all running jobs I see 100s of real-time searches "<CODE>|</CODE>" (a single pipe) with no start-time or end-time by the user Admin. Initially, I thought this may be related to work on a real-time dashboard and post-process searches, but the dashboard is in a different app than where these mysterious searches are reportedly running.</P> <P>To resolve the issue, I restart Splunk and delete the searches out of the dispatch directory. Simply trying to stop/delete them from the job management app will not work. </P> <P>Is there any way to determine what is causing this? Could this be a bug in version 6?</P> Sun, 06 Oct 2013 15:30:23 GMT https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84569#M21554 sc0tt 2013-10-06T15:30:23Z https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84570#M21555 <P>Just curious on the job management app - whats it?</P> Fri, 20 Dec 2013 16:01:57 GMT https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84570#M21555 splunkears 2013-12-20T16:01:57Z https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84571#M21556 <P>I was referring to the job manager; not really a separate app.</P> Mon, 06 Jan 2014 14:44:27 GMT https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84571#M21556 sc0tt 2014-01-06T14:44:27Z https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84572#M21557 <P>I would suspect that this could be one of a couple of things (that I can think of).<BR /> How tight is your control of searches on the box? Is it possible that users have created lots of subsearches or real time searches and used the map command? I can't recall of the top of my head how this would appear in the job manager but it might fit the pattern.</P> <P>Another option is that you have a couple of dashboards which have some oddly created searches that are impacting in v6 but weren't in v5? Can you do a search through your audit/internal logs to see where these searches are firing from.</P> Mon, 06 Jan 2014 19:07:28 GMT https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84572#M21557 Drainy 2014-01-06T19:07:28Z https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84573#M21558 <P>Good questions and suggestions. I'll dig into those deeper to see if I can isolate the issue. I'll follow up.</P> Tue, 14 Jan 2014 16:37:58 GMT https://community.splunk.com/t5/Splunk-Search/Mysterious-realtime-search-generating-100s-of-jobs/m-p/84573#M21558 sc0tt 2014-01-14T16:37:58Z