topic Splunkweb - slow startup in Security

Splunkweb - slow startup

kochera — Thu, 05 Jan 2012 13:39:53 GMT

Hi,

splunkweb takes ages to start. it even runs into an internal timeout and only with some tweaking (touch /appl/splunk/var/run/splunk/splunkweb.pid) it finally starts up after 2-10 minutes. We experience the same on 6 different Splunk instances. Find below an example which represents the second restart of a splunk instance. This one is faster as the first one but it still takes too long.

Without tweaking (service log):
Starting splunk server daemon (splunkd)...
Done.
Starting splunkweb...
Timed out waiting for splunkweb to start

With tweaking (web_service.log) :
2012-01-05 14:08:26,875 INFO [4f05a0c7da160f4d0] root:243 - Enabling SSL
2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Serving on 0.0.0.0:443
2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Bus STARTED

Any idea what migth cause this behaviour?

OS: Solaris 10
HW: x86
Splunk: 4.2.5

Re: Splunkweb - slow startup

LCM — Thu, 05 Jan 2012 14:45:46 GMT

I spotted slow startup once after I was cleaning up my buckets (splunk clean all). After that Splunk was checking/re-config all buckets (and I have/had a load of buckets) which took some 40mins for startup! But this happened only once after the cleanup!

Re: Splunkweb - slow startup

bojanz — Thu, 05 Jan 2012 15:36:11 GMT

This could indicate issues with DNS, so I would suggest that you check if the DNS server(s) Splunk server is using (/etc/resolv.conf) work correctly.

Re: Splunkweb - slow startup

kochera — Fri, 06 Jan 2012 06:27:03 GMT

Hi,

we have the problem with each restart. Time drops after multiple restarts within a short time (e.g. 2 hours).

cheers,
andy

Re: Splunkweb - slow startup

kochera — Fri, 06 Jan 2012 06:29:01 GMT

Hi,

DNS configuration is correct.

cheers,
Andy

Re: Splunkweb - slow startup

joelzyla — Fri, 22 Feb 2013 13:28:31 GMT

How are you "tweaking"?

Re: Splunkweb - slow startup

Ayn — Fri, 22 Feb 2013 13:31:18 GMT

I've seen this every now and then on systems with improper DNS settings.

Re: Splunkweb - slow startup

ashutoshab — Fri, 22 Sep 2017 18:23:18 GMT

I think your Splunk Web is trying to resolve 'splunk.com' while doing a SPLUNK START OR SPLUNK RESTART. You can try and provide connectivity to Public Internet so that splunk can resolve and connect to splunk.com. I faced this same issue recently and found out by doing a tail -f /opt/splunk/var/log/splunk/splunkd.log while giving splunk restart command, that in some way, splunk is trying to resolve splunk.com and is failing as the system does not have Connectivity to public Internet.

Check to see if the connectivity is proper and then give the splunk restart command, this solved the problem for me.