https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285539#M86406 <P>thank u so much ! it works !!</P> Thu, 26 Jan 2017 02:03:26 GMT shwesinhan 2017-01-26T02:03:26Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285532#M86399 <P>Hi everyone, I have a big issue.</P> <P>Since Friday, my single node Splunk instance stopped indexing data. I was in the process of deleting and removing old app files, and I think that I accidentally disabled a default app. Before beginning the process, I created a diag file, and I have already replaced my apps folder with the folder as it was before I started to create any mess, but unluckily, Splunk still doesn't index any data.</P> <P>I'm unable to search <CODE>index=_internal</CODE>, there are zero logs. <BR /> Looking in splunkd.log, I can't find any errors related with this problem. The logs that Splunk is suppose to read are still being collected by syslog, but they aren't being indexed. </P> <P>I don't really know what else can I do. The system folder is fine, I really checked everything. <BR /> The only difference I reported was on "server control". I'm unable to restart Splunk from the GUI and a message is there: </P> <PRE><CODE>The Splunkweb interface has been disabled. You must restart Splunk via the command line (or services control panel). </CODE></PRE> <P>I already tried to run the command <A href="http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/StartSplunk">http://docs.splunk.com/Documentation/Splunk/6.2.0/Admin/StartSplunk</A> here to start and stop splunkweb and the status said that Splunk is correctly running.</P> <P>The only errors message in splunk.d that I can find are: </P> <PRE><CODE> 12-20-2015 14:01:01.929 +0000 WARN ExecProcessor - Streaming XML data: Expected tag "event", instead received "error". 12-20-2015 14:01:01.929 +0000 WARN ExecProcessor - Streaming XML data: Expected tag "event", instead received "message". </CODE></PRE> <P>But I don't believe that this is related with the stopped indexing. I have S.o.S and Splunk Health check installed, and both doesn't report any issues...</P> <P>Please to help me, I don't really know what else can I do. </P> Mon, 21 Dec 2015 11:25:39 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285532#M86399 Federica_92 2015-12-21T11:25:39Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285533#M86400 <P>Sounds like your license expired. Is everything OK in the license tab?</P> Mon, 21 Dec 2015 13:45:41 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285533#M86400 jplumsdaine22 2015-12-21T13:45:41Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285534#M86401 <P>yeah, everything is ok...</P> Mon, 21 Dec 2015 13:58:40 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285534#M86401 Federica_92 2015-12-21T13:58:40Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285535#M86402 <P>Ah. Well I would get in touch with Splunk Support pronto</P> Mon, 21 Dec 2015 14:27:28 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285535#M86402 jplumsdaine22 2015-12-21T14:27:28Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285536#M86403 <P>I did it before, but it takes really long...I'am still waiting that they reply for a case 3 weeks old..</P> Mon, 21 Dec 2015 14:31:13 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285536#M86403 Federica_92 2015-12-21T14:31:13Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285537#M86404 <P>Hopefully someone on the forum can help you faster. It's weird you don't see anything for index=_internal. Its like your old indexes are gone. </P> Mon, 21 Dec 2015 14:40:25 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285537#M86404 jplumsdaine22 2015-12-21T14:40:25Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285538#M86405 <P>Check if you enabled any forwarder app; goto $SPLUNK_HOME/etc/apps and check any <CODE>app.conf</CODE> inside <CODE>SplunkForwarder</CODE> and <CODE>SplunkLightForwarder</CODE> for some setting like <CODE>state = enabled</CODE> or <CODE>state = 1</CODE>. If they are enabled, disable them and restart - Happy Splunking <span class="lia-unicode-emoji" title=":slightly_smiling_face:">🙂</span></P> Mon, 21 Dec 2015 19:35:32 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285538#M86405 MuS 2015-12-21T19:35:32Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285539#M86406 <P>thank u so much ! it works !!</P> Thu, 26 Jan 2017 02:03:26 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285539#M86406 shwesinhan 2017-01-26T02:03:26Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285540#M86407 <P>What about the same errors on search head in distributed environment?</P> Wed, 21 Jun 2017 13:11:47 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285540#M86407 thirumalreddyb 2017-06-21T13:11:47Z https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285541#M86408 <P>Are you referring to the message about the <CODE>Splunkweb interface</CODE> or the <CODE>Streaming XML</CODE>?</P> Wed, 21 Jun 2017 19:45:27 GMT https://community.splunk.com/t5/Splunk-Search/How-do-I-troubleshoot-why-Splunk-has-stopped-indexing-data-and/m-p/285541#M86408 MuS 2017-06-21T19:45:27Z