Monitoring Splunk

503 Service unavailable - Is my splunkd crashed ?

chimbudp
Contributor

Could not able to access Splunkweb.
ERROR : 503 Service unavailable
The splunkd daemon cannot be reached by splunkweb. Check that there are no blocked network ports or that splunkd is still running.

Splunkd , Splunkweb - Services are running fine. VERIFIED

How can i troubleshoot this issue ?
Please help

Tags (1)
0 Karma
1 Solution

krugger
Communicator

There sure seems to be a problem. Usually get that when splunk is restarting and therefore is not ready to handle my web requests.

Run in the shell:
splunk status

Should say something like:
splunkd is running (PID: 16691).
splunk helpers are running (PIDs: 16692).
splunkweb is running (PID: 16991).

Have a look at /opt/splunk/var/log/splunk/splunkd.log.

View solution in original post

chskm
Path Finder

The issue will raise at the time of splunk restart or else in some times due to network outage. Try logging into splunkweb for particular server. If you can able to login ten everything is working properly. Even you can check by loggin into indexers. ( cluster master to check the splunk is up or not)

0 Karma

chimbudp
Contributor

Instance of splunk is running already and it enters into a CLOSE_WAIT state.
So , we need to kill the previously existing Splunk instance in the TASK MANAGER.

Its working now.

0 Karma

krugger
Communicator

There sure seems to be a problem. Usually get that when splunk is restarting and therefore is not ready to handle my web requests.

Run in the shell:
splunk status

Should say something like:
splunkd is running (PID: 16691).
splunk helpers are running (PIDs: 16692).
splunkweb is running (PID: 16991).

Have a look at /opt/splunk/var/log/splunk/splunkd.log.

mehdihammami
Engager

03-29-2021 04:00:35.158 +0100 WARN ProcessTracker - executable=splunk-optimize failed to start reason='': Média protégé en écriture. 

i have the same probleme in splunk on WINDOWS 10, when i had a look in splunk/var/log/splunk/splunkd.log he told splunk failed to start ( 03-29-2021 04:00:35.158 +0100 WARN ProcessTracker - executable=splunk-optimize failed to start reason='': Média protégé en écriture. ) 

 

0 Karma

stefano_guidoba
Communicator

My first suggestion is to check splunk logs at $SPLUNK_HOME/var/log. Do they suggest anything?

0 Karma

chskm
Path Finder

The issue will raise at the time of splunk restart or else in some times due to network outage. Try logging into splunkweb for particular server. If you can able to login ten everything is working properly. Even you can check by loggin into indexers. ( cluster master to check the splunk is up or not)

0 Karma
Get Updates on the Splunk Community!

Extending Observability Content to Splunk Cloud

Watch Now!   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to leverage ...

More Control Over Your Monitoring Costs with Archived Metrics!

What if there was a way you could keep all the metrics data you need while saving on storage costs?This is now ...

New in Observability Cloud - Explicit Bucket Histograms

Splunk introduces native support for histograms as a metric data type within Observability Cloud with Explicit ...