Splunk Enterprise
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Ask a Question

KVstore unable to start after upgrade to Splunk Enterprise 9.4

gloom
Loves-to-Learn
a week ago

Hi,

After completing the upgrade from Splunk Enterprise version 9.3.2 to v9.4 the KVstore will no longer start. Splunk has yet to do the KVstore upgrade to v7 as the KVstore cannot start. We were already on 4.2 wiredtiger.

The is no [kvstore] stanza in server.conf so everything should be default.

The relavent lines from splunkd.log are:

 

 

INFO  KVStoreConfigurationProvider [9192 MainThread] - Since x509 is not enabled - using a default config from [sslConfig] for Mongod mTLS authentication
WARN  KVStoreConfigurationProvider [9192 MainThread] - Action scheduled, but event loop is not ready yet
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Starting mongod with executable name=mongod-4.2.exe version=kvstore version 4.2
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --dbpath C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo 
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --storageEngine wiredTiger
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using cacheSize=1.65GB
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --port 8191
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --timeStampFormat iso8601-utc
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --oplogSize 200
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --keyFile C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo\splunk.key
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --setParameter enableLocalhostAuthBypass=0
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --setParameter oplogFetcherSteadyStateMaxFetcherRestarts=0
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --replSet 4EA2F2AF-2584-4BB0-A2C4-414E7CB68BC2
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --bind_ip=0.0.0.0 (all ipv4 addresses)
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslCAFile C:\Program Files\Splunk\etc\auth\cacert.pem
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --tlsAllowConnectionsWithoutCertificates for version 4.2
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslMode requireSSL
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslAllowInvalidHostnames
WARN  KVStoreConfigurationProvider [9192 MainThread] - Action scheduled, but event loop is not ready yet
INFO  KVStoreConfigurationProvider [9192 MainThread] - "SAML cert db" registration with KVStore successful
INFO  KVStoreConfigurationProvider [9192 MainThread] - "Auth cert db" registration with KVStore successful
INFO  KVStoreConfigurationProvider [9192 MainThread] - "JsonWebToken Manager" registration with KVStore successful
INFO  KVStoreBackupRestore [1436 KVStoreBackupThread] - thread started.
INFO  KVStoreConfigurationProvider [9192 MainThread] - "Certificate Manager" registration with KVStore successful
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Found an existing PFX certificate
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslCertificateSelector subject=SplunkServerDefaultCert
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslAllowInvalidCertificates
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --tlsDisabledProtocols noTLS1_0,noTLS1_1
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --sslCipherConfig ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDH-ECDSA-AES256-GCM-SHA384:ECDH-ECDSA-AES128-GCM-SHA256:ECDH-ECDSA-AES128-SHA256:AES256-GCM-SHA384:AES128-GCM-SHA256:AES128-SHA256
INFO  MongodRunner [7668 KVStoreConfigurationThread] - Using mongod command line --noscripting
WARN  MongoClient [7668 KVStoreConfigurationThread] - Disabling TLS hostname validation for localhost
ERROR MongodRunner [5692 MongodLogThread] - mongod exited abnormally (exit code 14, status: exited with code 14) - look at mongod.log to investigate.
ERROR KVStoreBulletinBoardManager [5692 MongodLogThread] - KV Store process terminated abnormally (exit code 14, status exited with code 14). See mongod.log and splunkd.log for details.
WARN  KVStoreConfigurationProvider [5692 MongodLogThread] - Action scheduled, but event loop is not ready yet
ERROR KVStoreBulletinBoardManager [5692 MongodLogThread] - KV Store changed status to failed. KVStore process terminated..
ERROR KVStoreConfigurationProvider [7668 KVStoreConfigurationThread] - Failed to start mongod on first attempt reason=KVStore service will not start because kvstore process terminated
ERROR KVStoreConfigurationProvider [7668 KVStoreConfigurationThread] - Could not start mongo instance. Initialization failed.
ERROR KVStoreBulletinBoardManager [7668 KVStoreConfigurationThread] - Failed to start KV Store process. See mongod.log and splunkd.log for details.
INFO  KVStoreConfigurationProvider [7668 KVStoreConfigurationThread] - Mongod service shutting down

 

 

mogod.log contains the following:

 

W  CONTROL  [main] Option: sslMode is deprecated. Please use tlsMode instead.
W  CONTROL  [main] Option: sslCAFile is deprecated. Please use tlsCAFile instead.
W  CONTROL  [main] Option: sslCipherConfig is deprecated. Please use tlsCipherConfig instead.
W  CONTROL  [main] Option: sslAllowInvalidHostnames is deprecated. Please use tlsAllowInvalidHostnames instead.
W  CONTROL  [main] Option: sslAllowInvalidCertificates is deprecated. Please use tlsAllowInvalidCertificates instead.
W  CONTROL  [main] Option: sslCertificateSelector is deprecated. Please use tlsCertificateSelector instead.
W  CONTROL  [main] net.tls.tlsCipherConfig is deprecated. It will be removed in a future release.
W  NETWORK  [main] Mixing certs from the system certificate store and PEM files. This may produced unexpected results.
W  NETWORK  [main] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
W  NETWORK  [main] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
W  NETWORK  [main] Server certificate has no compatible Subject Alternative Name. This may prevent TLS clients from connecting
W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
W  NETWORK  [main] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
W  ASIO     [main] No TransportLayer configured during NetworkInterface startup
W  NETWORK  [main] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
I  CONTROL  [initandlisten] MongoDB starting : pid=4640 port=8191 dbpath=C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo 64-bit host=[redacted]
I  CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
I  CONTROL  [initandlisten] db version v4.2.24
I  CONTROL  [initandlisten] git version: 5e4ec1d24431fcdd28b579a024c5c801b8cde4e2
I  CONTROL  [initandlisten] allocator: tcmalloc
I  CONTROL  [initandlisten] modules: enterprise 
I  CONTROL  [initandlisten] build environment:
I  CONTROL  [initandlisten]     distmod: windows-64
I  CONTROL  [initandlisten]     distarch: x86_64
I  CONTROL  [initandlisten]     target_arch: x86_64
I  CONTROL  [initandlisten] options: { net: { bindIp: "0.0.0.0", port: 8191, tls: { CAFile: "C:\Program Files\Splunk\etc\auth\cacert.pem", allowConnectionsWithoutCertificates: true, allowInvalidCertificates: true, allowInvalidHostnames: true, certificateSelector: "subject=SplunkServerDefaultCert", disabledProtocols: "noTLS1_0,noTLS1_1", mode: "requireTLS", tlsCipherConfig: "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RS..." } }, replication: { oplogSizeMB: 200, replSet: "4EA2F2AF-2584-4BB0-A2C4-414E7CB68BC2" }, security: { javascriptEnabled: false, keyFile: "C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo\splunk.key" }, setParameter: { enableLocalhostAuthBypass: "0", oplogFetcherSteadyStateMaxFetcherRestarts: "0" }, storage: { dbPath: "C:\Program Files\Splunk\var\lib\splunk\kvstore\mongo", engine: "wiredTiger", wiredTiger: { engineConfig: { cacheSizeGB: 1.65 } } }, systemLog: { timeStampFormat: "iso8601-utc" } }
W  NETWORK  [initandlisten] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
W  NETWORK  [initandlisten] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1689M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
W  STORAGE  [initandlisten] Failed to start up WiredTiger under any compatibility version.
F  STORAGE  [initandlisten] Reason: 129: Operation not supported
F  -        [initandlisten] Fatal Assertion 28595 at src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp 928
F  -        [initandlisten] \n\n***aborting after fassert() failure\n\n

 

 Does anyone have any idea how to resolve this?

Thanks,

Labels (1)
Labels
0 Karma
Reply

apietersen
Contributor
Wednesday

Hi,

We have the same issue here. Upgraded from Splunk Ent. v9.3.2 to V9.40 , running Windows 2019 server. The Kvstore process not running also effect on Splunk Secure Gateway (SSG/Splunk Mobile), Dastboard Studio, (and i think Edge Hub etc). 😞
Yes, looked in mongod.log and splunkd.log but not a bit wiser!

apietersen_0-1736943083713.png

apietersen_1-1736945667034.png

See below some lines in my mongod.log :

targetMinOS: Windows 7/Windows Server 2008 R2 - ???
this build only supports versions up to 4, and the file is version 5: - ??

 

 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] MongoDB starting : pid=2224 port=8191 dbpath=D:\Program Files\Splunk\var\lib\splunk\kvstore\mongo 64-bit host=Gozer2
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] db version v4.2.24
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] git version: 5e4ec1d24431fcdd28b579a024c5c801b8cde4e2
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] allocator: tcmalloc
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] modules: enterprise 
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten] build environment:
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten]     distmod: windows-64
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten]     distarch: x86_64
 2025-01-15T14:45:22.046Z I  CONTROL  [initandlisten]     target_arch: x86_64
 2025-01-15T14:45:22.047Z I  CONTROL  [initandlisten] options: { net: { bindIp: "0.0.0.0", port: 8191, tls: { CAFile: "D:\Program Files\Splunk\etc\auth\cacert.pem", allowConnectionsWithoutCertificates: true, allowInvalidCertificates: true, allowInvalidHostnames: true, certificateSelector: "subject=SplunkServerDefaultCert", disabledProtocols: "noTLS1_0,noTLS1_1", mode: "requireTLS", tlsCipherConfig: "ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-SHA384:ECDHE-RS..." } }, replication: { oplogSizeMB: 200, replSet: "102D93C2-E5B9-4347-88CA-59FB829D92E1" }, security: { javascriptEnabled: false, keyFile: "D:\Program Files\Splunk\var\lib\splunk\kvstore\mongo\splunk.key" }, setParameter: { enableLocalhostAuthBypass: "0", oplogFetcherSteadyStateMaxFetcherRestarts: "0" }, storage: { dbPath: "D:\Program Files\Splunk\var\lib\splunk\kvstore\mongo", engine: "wiredTiger", wiredTiger: { engineConfig: { cacheSizeGB: 4.65 } } }, systemLog: { timeStampFormat: "iso8601-utc" } }
 2025-01-15T14:45:22.048Z W  NETWORK  [initandlisten] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
 2025-01-15T14:45:22.048Z W  NETWORK  [initandlisten] sslCipherConfig parameter is not supported with Windows SChannel and is ignored.
 2025-01-15T14:45:22.049Z I  STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=4761M,cache_overflow=(file_max=0M),session_max=33000,eviction=(threads_min=4,threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000,close_scan_interval=10,close_handle_minimum=250),statistics_log=(wait=0),verbose=[recovery_progress,checkpoint_progress],
 2025-01-15T14:45:22.083Z E  STORAGE  [initandlisten] WiredTiger error (-31802) [1736952322:82769][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error Raw: [1736952322:82769][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.100Z E  STORAGE  [initandlisten] WiredTiger error (-31802) [1736952322:100690][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error Raw: [1736952322:100690][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.116Z E  STORAGE  [initandlisten] WiredTiger error (-31802) [1736952322:115624][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error Raw: [1736952322:115624][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.150Z E  STORAGE  [initandlisten] WiredTiger error (-31802) [1736952322:149476][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error Raw: [1736952322:149476][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.175Z E  STORAGE  [initandlisten] WiredTiger error (-31802) [1736952322:175362][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error Raw: [1736952322:175362][2224:140709387064240], connection: __log_open_verify, 925: unsupported WiredTiger file version: this build only supports versions up to 4, and the file is version 5: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.179Z W  STORAGE  [initandlisten] Failed to start up WiredTiger under any compatibility version.
 2025-01-15T14:45:22.179Z F  STORAGE  [initandlisten] Reason: -31802: WT_ERROR: non-specific WiredTiger error
 2025-01-15T14:45:22.179Z F  -        [initandlisten] Fatal Assertion 28595 at src\mongo\db\storage\wiredtiger\wiredtiger_kv_engine.cpp 928
 2025-01-15T14:45:22.179Z F  -        [initandlisten] \n\n***aborting after fassert() failure\n\n

 

Some lines from my Slunkd.log:

01-15-2025 15:57:57.139 +0100 INFO  TailReader [7248 tailreader0] - Batch input finished reading file='D:\Program Files\Splunk\var\spool\splunk\tracker.log'
01-15-2025 15:57:57.467 +0100 ERROR KVStorageProvider [5552 TcpChannelThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:57:57.467 +0100 ERROR KVStoreAdminHandler [5552 TcpChannelThread] - An error occurred.
01-15-2025 15:58:03.592 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:10.645 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:17.723 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:24.745 +0100 WARN  ExecProcessor [10156 ExecProcessor] - message from ""D:\Program Files\Splunk\bin\splunk-regmon.exe""  BundlesUtil - D:\Program Files\Splunk\etc\system\metadata\local.meta already exists but with different casing: D:\Program Files\splunk\etc\system\metadata\local.meta
01-15-2025 15:58:24.792 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:27.307 +0100 INFO  TailReader [7248 tailreader0] - Batch input finished reading file='D:\Program Files\Splunk\var\spool\splunk\tracker.log'
01-15-2025 15:58:31.865 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:38.929 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:46.000 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:53.049 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:58:56.617 +0100 INFO  TailReader [7248 tailreader0] - Batch input finished reading file='D:\Program Files\Splunk\var\spool\splunk\tracker.log'
01-15-2025 15:59:00.117 +0100 ERROR KVStorageProvider [924 KVStoreUpgradeStartupThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:59:01.460 +0100 ERROR KVStorageProvider [5608 TcpChannelThread] - An error occurred during the last operation ('replSetGetStatus', domain: '15', code: '13053'): No suitable servers found (`serverSelectionTryOnce` set): [connection closed calling hello on 'gozer2:8191']
01-15-2025 15:59:01.460 +0100 ERROR KVStoreAdminHandler [5608 TcpChannelThread] - An error occurred.
0 Karma
Reply

MaverickT
Communicator
a week ago

Maybe this can be solution to your challenge: https://community.splunk.com/t5/Deployment-Architecture/KVStore-does-not-start-when-running-Splunk-9...

0 Karma
Reply

gloom
Loves-to-Learn
a week ago

This seems confusing, as Splunk hasn't attempted to do the mongodb upgrade yet, I would expect it to fail after the upgrade if this was the case?

 

Edit: I ran HWinfo on the box, its showing AVX, AVX2 and AVX-512 supported, so I don't think this is the issue.

0 Karma
Reply

isoutamo
SplunkTrust
SplunkTrust
a week ago

Hi

Based on log you are running unsupported OS.

 

 CONTROL  [initandlisten] targetMinOS: Windows 7/Windows Server 2008 R2

 

On Windows operating systems oldest supported version is Win 2019 or Win 10.

r. Ismo 

0 Karma
Reply

gloom
Loves-to-Learn
a week ago

Thats incorrect, its a server 2022 box.

0 Karma
Reply
Get Updates on the Splunk Community!

Now Available: Cisco Talos Threat Intelligence Integrations for Splunk Security Cloud ...

At .conf24, we shared that we were in the process of integrating Cisco Talos threat intelligence into Splunk ...

Preparing your Splunk Environment for OpenSSL3

The Splunk platform will transition to OpenSSL version 3 in a future release. Actions are required to prepare ...

Easily Improve Agent Saturation with the Splunk Add-on for OpenTelemetry Collector

Agent Saturation What and Whys In application performance monitoring, saturation is defined as the total load ...