https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23053#M778 <P>Some wired thing got happen into the server. splunkd was running but the splunk web was down. A restart made everything normal. But i need to know why the splunk web was down. Have got nothing suspicious in splunkd.log and web_server.log.</P> <P>Is there any way to know why the wplunk web was down??</P> Thu, 07 Feb 2013 14:10:15 GMT Splunk_U 2013-02-07T14:10:15Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23053#M778 <P>Some wired thing got happen into the server. splunkd was running but the splunk web was down. A restart made everything normal. But i need to know why the splunk web was down. Have got nothing suspicious in splunkd.log and web_server.log.</P> <P>Is there any way to know why the wplunk web was down??</P> Thu, 07 Feb 2013 14:10:15 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23053#M778 Splunk_U 2013-02-07T14:10:15Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23054#M779 <P>Check the _internal index for the logs in web_service.log.<BR /> Do you see anything prior to the stopping ?</P> <P>Otherwise, if you are on linux, check the /var/log/messages for any "Out Of Memory / OOM" events, the system can kill a process.</P> Mon, 28 Sep 2020 13:15:16 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23054#M779 yannK 2020-09-28T13:15:16Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23055#M780 <P>I have checked the messages. I have found that...<BR /> sisidsdaemon invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0<BR /> and after that..<BR /> Out of memory: Kill process 3936 (python) score 730 or sacrifice child<BR /> So looks like splunk web got down due to out of memeory...</P> Mon, 28 Sep 2020 13:15:18 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23055#M780 Splunk_U 2020-09-28T13:15:18Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23056#M781 <P>I have understood that the root cause is Out of memory but is there any process to check the memory consumption by the search in Splunk?</P> Mon, 11 Feb 2013 15:59:22 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23056#M781 Splunk_U 2013-02-11T15:59:22Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23057#M782 <P>If my search is having event_count=20000 is it eating much memory than the event_count=100???</P> Mon, 28 Sep 2020 13:17:35 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23057#M782 Splunk_U 2020-09-28T13:17:35Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23058#M783 <P>the sos app and the ps_sos scripts, will show you cpu/memory of expensive searches<BR /> <A href="http://splunk-base.splunk.com/apps/29008/sos-splunk-on-splunk">http://splunk-base.splunk.com/apps/29008/sos-splunk-on-splunk</A></P> Mon, 11 Feb 2013 19:14:41 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23058#M783 yannK 2013-02-11T19:14:41Z https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23059#M784 <P>Yeah that I know...but Sos was not installed in the splunk server when the issue got happened...thats why asking that by seeing event_count, total_run_time etc can we understand the memory consumption...or least can we understand the relation between memory consumption with evvent_count or total_run_time?</P> Mon, 28 Sep 2020 13:17:43 GMT https://community.splunk.com/t5/Security/splunk-web-got-down/m-p/23059#M784 Splunk_U 2020-09-28T13:17:43Z