Splunk User Behavior Analytics

Splunk UBA is down

snisaxena
Loves-to-Learn

Splunk UBA search head is down.

Even after restarting ui services, status is shown as active in CLI but GUI is not available.

Commands used to stop/start ui service:

sudo service caspida-ui stop
 sudo service caspida-ui start

 

Status when checked in CLI:

caspida-ui.service
Loaded: loaded (/etc/init.d/caspida-ui; bad; vendor preset: enabled)
Active: active (exited) since Fri 2021-09-03 05:53:12 UTC; 6min ago

I also tried rebooting the VM, but it doesn't help.

 

Can I please get a suggestion around how to fix this?

Labels (3)
0 Karma

lakshman239
SplunkTrust
SplunkTrust

did this setup work in the past? If so, has there been any changes to IP/host/dns resolution and/or firewall/connectivity? looks like connectivity/resolution issue

0 Karma

snisaxena
Loves-to-Learn

@lakshman239  I suspect so too. However, there is no confirmation from network team regarding any connection changes wrt firewall, etc.

0 Karma

lakshman239
SplunkTrust
SplunkTrust

@snisaxena  One option would be stop and start all services, so they start gracefully. Pls refer to - https://docs.splunk.com/Documentation/UBA/5.0.4.1/Admin/CLICommands 

0 Karma

snisaxena
Loves-to-Learn

@lakshman239I ran /opt/caspida/bin/Caspida stop-all and it has been running since more than 2 hours now.
I tried to exit and run /opt/caspida/bin/Caspida start-all. It was aborted with below message:

failed to check/update system configuration: aborting. see /var/vcap/sys/log/caspida/caspida.out

0 Karma

lakshman239
SplunkTrust
SplunkTrust

stop-all running for long time does indicate an underlying issue in the cluster.

Have you run the pre-check and post health checks using the latest available scripts? If not, please run them and perhaps raise a case with support attaching the output.

0 Karma

snisaxena
Loves-to-Learn

@lakshman239  I did run a health check before running stop-all and observed below error:

ui connect: <hostname> <= curl failed to ui <hostname>
curl: (7) Failed to connect to <hostname> port 443: Connection refused
ui connect: sc2-splunk-uba-1 <= curl failed to ui <hostname>
curl: (7) Failed to connect to <hostname> port 443: Connection refused

0 Karma
Get Updates on the Splunk Community!

Splunk Education - Fast Start Program!

Welcome to Splunk Education! Splunk training programs are designed to enable you to get started quickly and ...

Five Subtly Different Ways of Adding Manual Instrumentation in Java

You can find the code of this example on GitHub here. Please feel free to star the repository to keep in ...

New Splunk APM Enhancements Help Troubleshoot Your MySQL and NoSQL Databases Faster

Splunk Observability has two new enhancements to make it quicker and easier to troubleshoot slow or frequently ...