Monitoring Splunk

503 Service unavailable - Is my splunkd crashed ?

chimbudp
Contributor

Could not able to access Splunkweb.
ERROR : 503 Service unavailable
The splunkd daemon cannot be reached by splunkweb. Check that there are no blocked network ports or that splunkd is still running.

Splunkd , Splunkweb - Services are running fine. VERIFIED

How can i troubleshoot this issue ?
Please help

Tags (1)
0 Karma
1 Solution

krugger
Communicator

There sure seems to be a problem. Usually get that when splunk is restarting and therefore is not ready to handle my web requests.

Run in the shell:
splunk status

Should say something like:
splunkd is running (PID: 16691).
splunk helpers are running (PIDs: 16692).
splunkweb is running (PID: 16991).

Have a look at /opt/splunk/var/log/splunk/splunkd.log.

View solution in original post

chskm
Path Finder

The issue will raise at the time of splunk restart or else in some times due to network outage. Try logging into splunkweb for particular server. If you can able to login ten everything is working properly. Even you can check by loggin into indexers. ( cluster master to check the splunk is up or not)

0 Karma

chimbudp
Contributor

Instance of splunk is running already and it enters into a CLOSE_WAIT state.
So , we need to kill the previously existing Splunk instance in the TASK MANAGER.

Its working now.

0 Karma

krugger
Communicator

There sure seems to be a problem. Usually get that when splunk is restarting and therefore is not ready to handle my web requests.

Run in the shell:
splunk status

Should say something like:
splunkd is running (PID: 16691).
splunk helpers are running (PIDs: 16692).
splunkweb is running (PID: 16991).

Have a look at /opt/splunk/var/log/splunk/splunkd.log.

mehdihammami
Engager

03-29-2021 04:00:35.158 +0100 WARN ProcessTracker - executable=splunk-optimize failed to start reason='': Média protégé en écriture. 

i have the same probleme in splunk on WINDOWS 10, when i had a look in splunk/var/log/splunk/splunkd.log he told splunk failed to start ( 03-29-2021 04:00:35.158 +0100 WARN ProcessTracker - executable=splunk-optimize failed to start reason='': Média protégé en écriture. ) 

 

0 Karma

stefano_guidoba
Communicator

My first suggestion is to check splunk logs at $SPLUNK_HOME/var/log. Do they suggest anything?

0 Karma

chskm
Path Finder

The issue will raise at the time of splunk restart or else in some times due to network outage. Try logging into splunkweb for particular server. If you can able to login ten everything is working properly. Even you can check by loggin into indexers. ( cluster master to check the splunk is up or not)

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Network to App: Observability Unlocked [May & June Series]

In today’s digital landscape, your environment is no longer confined to the data center. It spans complex ...

SPL2 Deep Dives, AppDynamics Integrations, SAML Made Simple and Much More on Splunk ...

Splunk Lantern is Splunk’s customer success center that provides practical guidance from Splunk experts on key ...

[Puzzles] Solve, Learn, Repeat: Matching cron expressions

This puzzle (first published here) is based on matching timestamps to cron expressions.All the timestamps ...