Monitoring Splunk

Splunk web down adter debug/refresh

faustf
Communicator

Hi guys,
I've installed a Splunk enterprise 6.5.2 and some Splunk applications.
It's a while that when I try to click on refresh button ( splunkserver:port/debug/refresh ) after a couples of minutes the web interface is not reachable anymore and I've to restart splunk.
In splunk.log files there are some errors related to the Splunk applications installed but they seem to be not important.
Also I see the following:

ERROR KVStoreBulletinBoardManager - KV Store changed status to failed. KVStore process terminated.
05-04-2017 08:39:23.487 +0200 ERROR KVStorageProvider - An error occurred during the last operation ('saveBatchData', domain: '2', code: '5'): Failed to connect to target host: 127.0.0.1:8191

What Can I do?

Tags (2)
0 Karma

bohanlon_splunk
Splunk Employee
Splunk Employee

Sounds like there is something going wrong with the KV store (mongo) during this operation.

First place I would look is in mongod.log. ($splunk_home/var/log/splunk/mongod.log)

Second thing I would do is make sure your KVstore is loading ok normally. I would start by checking stuff like certs which KV store uses to self-validate:
https://answers.splunk.com/answers/404979/trying-to-run-the-distributed-management-console-g.html

Third thing I would do is confirm you actually use KVstore here, and that this message is not a red herring.

0 Karma
Get Updates on the Splunk Community!

Why You Can't Miss .conf25: Unleashing the Power of Agentic AI with Splunk & Cisco

The Defining Technology Movement of Our Lifetime The advent of agentic AI is arguably the defining technology ...

Deep Dive into Federated Analytics: Unlocking the Full Power of Your Security Data

In today’s complex digital landscape, security teams face increasing pressure to protect sprawling data across ...

Your summer travels continue with new course releases

Summer in the Northern hemisphere is in full swing, and is often a time to travel and explore. If your summer ...