Looking for a REST status OK for load balancer healthcheck

[davebo1896]

I'm looking for a basic heartbeat or healthcheck for search head status, in order to put behind a load balancer.
This should not require authentication.
The closest thing I've found is in the REST API /services/server/info , but that provides way more information than is needed.
Isn't there a simple call that will return "OK" ?

[MuS · Accepted solution]

Greetings from the future,

there is an app https://splunkbase.splunk.com/app/4395/ now that provides a REST API endpoint that the load balancer can check without authentication and without the need to enable the debug endpoints in web.conf.
It provides also an option to take the Splunk instance out of the load balancer group (the load balancer must support such a thing).

Hope this helps ...

cheers, MuS

[lpolo]

The approach I took is this one:

nc -zv splunkServer 514
found 0 associations
found 1 connections:
1: flags=82
outif en3
src x.x.x.x port 56747
dst x.x.x.x port 514
rank info not available
TCP aux info available
Connection to 172.24.236.98 port 514 [tcp/shell] succeeded!

You also could do:

https://yourSplunkIndexer:8089/services/server/info/server-info

Thanks,
Lp

[lpolo]

I facing the same situation.
What end point you end up using for health check?

Thanks,
Lp

[MuS]

@lpolo, look at the comment by davebo1896 you can use

 http[s]://YourSplunkServerName:YourSplunksPortHere/en-GB/debug/echo?ping=OK

and parse the output for the ping OK response.

cheers, MuS

[lpolo]

It does not work in 6.4.1. The response is:
Unauthorized to access this resource.

[MuS]

Sure it does 😉 You need to enable the debug endpoints first - they were turned off for security reasons. To enable them change the web.conf

enableWebDebug = true|false
* Controls the visibility of the debug endpoints (i.e., /debug/**splat).
* Defaults to false

[MuS]

Hi davebo1896,

you can add a bit of SPL to this REST call, like this:

| REST /services/server/info | stats count AS myCount | eval Status=if(myCount=="1", "OK" , "PANIK")| table Status

to get what you want or you just check the TCP port of the search head from load balancer. Because if the Web server from Splunk is down your search head is not usable by any user.

Hope this helps ...

cheers, MuS

[davebo1896]

We were hoping for something a bit more sophisticated than checking the TCP port. We want to know that the service is capable of returning something sensible, albeit simple.
For now we're looking at using /debug/echo?ping=OK and parsing for the OK, but that is less than optimal.
We may put an nginx server in front of Splunk Web to return a simple OK (or not), but that adds an extra layer that could cause false readings (both positive and negative).