Security

Splunkweb - slow startup

kochera
Communicator

Hi,

splunkweb takes ages to start. it even runs into an internal timeout and only with some tweaking (touch /appl/splunk/var/run/splunk/splunkweb.pid) it finally starts up after 2-10 minutes. We experience the same on 6 different Splunk instances. Find below an example which represents the second restart of a splunk instance. This one is faster as the first one but it still takes too long.

Without tweaking (service log):
Starting splunk server daemon (splunkd)...
Done.
Starting splunkweb...
Timed out waiting for splunkweb to start

With tweaking (web_service.log) :
2012-01-05 14:08:26,875 INFO [4f05a0c7da160f4d0] root:243 - Enabling SSL
2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Serving on 0.0.0.0:443
2012-01-05 14:10:57,546 INFO [4f05a0c7da160f4d0] root:133 - ENGINE: Bus STARTED

Any idea what migth cause this behaviour?

OS: Solaris 10
HW: x86
Splunk: 4.2.5

Tags (2)
0 Karma

ashutoshab
Communicator

I think your Splunk Web is trying to resolve 'splunk.com' while doing a SPLUNK START OR SPLUNK RESTART. You can try and provide connectivity to Public Internet so that splunk can resolve and connect to splunk.com. I faced this same issue recently and found out by doing a tail -f /opt/splunk/var/log/splunk/splunkd.log while giving splunk restart command, that in some way, splunk is trying to resolve splunk.com and is failing as the system does not have Connectivity to public Internet.

Check to see if the connectivity is proper and then give the splunk restart command, this solved the problem for me.

0 Karma

joelzyla
Explorer

How are you "tweaking"?

0 Karma

bojanz
Communicator

This could indicate issues with DNS, so I would suggest that you check if the DNS server(s) Splunk server is using (/etc/resolv.conf) work correctly.

0 Karma

kochera
Communicator

Hi,

DNS configuration is correct.

cheers,
Andy

0 Karma

LCM
Contributor

I spotted slow startup once after I was cleaning up my buckets (splunk clean all). After that Splunk was checking/re-config all buckets (and I have/had a load of buckets) which took some 40mins for startup! But this happened only once after the cleanup!

Ayn
Legend

I've seen this every now and then on systems with improper DNS settings.

0 Karma

kochera
Communicator

Hi,

we have the problem with each restart. Time drops after multiple restarts within a short time (e.g. 2 hours).

cheers,
andy

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Build the Future of Agentic AI: Join the Splunk Agentic Ops Hackathon

AI is changing how teams investigate incidents, detect threats, automate workflows, and build intelligent ...

[Puzzles] Solve, Learn, Repeat: Character substitutions with Regular Expressions

This challenge was first posted on Slack #puzzles channelFor BORE at .conf23, we had a puzzle question which ...

Splunk Community Badges!

  Hey everyone! Ready to earn some serious bragging rights in the community? Along with our existing badges ...