@livehybrid  Thanks for the response. Yes, some servers having custom certificates on those servers, we are having issue If I try changing to the default local certificate, then it works root@test02:/opt/splunk/bin# ./splunk cmd openssl verify -verbose -x509_strict -CAfile /opt/splunk/etc/auth/cacert.pem.default /opt/splunk/etc/auth/server.pem_old /opt/splunk/etc/auth/server.pem_old: OK root@test02:/opt/splunk/bin# root@test02:/opt/splunk/bin# root@test02:/opt/splunk/bin# root@test02:/opt/splunk/bin# ./splunk cmd openssl verify -verbose -x509_strict -CAfile /opt/splunk/etc/auth/cacert.pem /opt/splunk/etc/auth/server.pem error 20 at 0 depth lookup: unable to get local issuer certificate ./splunk cmd btool server list --debug kvstore /opt/splunk/etc/system/default/server.conf [kvstore] /opt/splunk/etc/system/default/server.conf clientConnectionPoolSize = 500 /opt/splunk/etc/system/default/server.conf clientConnectionTimeout = 10 /opt/splunk/etc/system/default/server.conf clientSocketTimeout = 300 /opt/splunk/etc/system/default/server.conf dbCursorOperationTimeout = 300 /opt/splunk/etc/system/default/server.conf dbPath = $SPLUNK_DB/kvstore /opt/splunk/etc/system/default/server.conf defaultKVStoreType = local /opt/splunk/etc/system/default/server.conf delayShutdownOnBackupRestoreInProgress = false /opt/splunk/etc/system/default/server.conf disabled = false /opt/splunk/etc/system/default/server.conf initAttempts = 300 /opt/splunk/etc/system/default/server.conf initialSyncMaxFetcherRestarts = 0 /opt/splunk/etc/system/default/server.conf kvstoreUpgradeCheckInterval = 5 /opt/splunk/etc/system/default/server.conf kvstoreUpgradeOnStartupDelay = 60 /opt/splunk/etc/system/default/server.conf kvstoreUpgradeOnStartupEnabled = true /opt/splunk/etc/system/default/server.conf kvstoreUpgradeOnStartupRetries = 2 /opt/splunk/etc/system/default/server.conf minSnapshotHistoryWindow = 5 /opt/splunk/etc/system/default/server.conf oplogSize = 1000 /opt/splunk/etc/system/default/server.conf percRAMForCache = 15 /opt/splunk/etc/system/default/server.conf port = 8191 /opt/splunk/etc/system/default/server.conf replicaset = splunkrs /opt/splunk/etc/system/default/server.conf replicationWriteTimeout = 1800 /opt/splunk/etc/system/default/server.conf shutdownTimeout = 100 /opt/splunk/etc/system/default/server.conf sslVerifyServerCert = false /opt/splunk/etc/system/default/server.conf sslVerifyServerName = false /opt/splunk/etc/system/default/server.conf storageEngine = wiredTiger /opt/splunk/etc/system/default/server.conf storageEngineMigration = false ... View more

Here are some internal logs: 2025-07-22T17:37:22.629Z I  NETWORK  [conn1078] Error receiving request from client: SSLHandshakeFailed: SSL peer certificate validation failed: self signed certificate in certificate chain. Ending connection from 127.0.0.1:43286 (connection id: 1078) 2025-07-22T17:37:22.629Z E  NETWORK  [conn1078] SSL peer certificate validation failed: self signed certificate in certificate chain T 2025-07-22T17:37:22.125Z I  NETWORK  [conn1077] Error receiving request from client: SSLHandshakeFailed: SSL peer certificate validation failed: self signed certificate in certificate chain. Ending connection from 127.0.0.1:43272 (connection id: 1077) h 2025-07-22T17:37:22.125Z E  NETWORK  [conn1077] SSL peer certificate validation failed: self signed certificate in certificate chain ... View more

@richgalloway  My understanding is that, if indextime is today so _time should have been the same since there is no delay during ingestion. Here is the default props.conf that comes with the SA-ldapsearch TA: (there is no transform.conf) [source::.../var/log/splunk/SA-ldapsearch.log] sourcetype = SA-ldapsearch [SA-ldapsearch] EXTRACT-vars = Level=.+, (?<log_source>Pid=.+, File=.+, Line=.+), (?<message>.*) ... View more

@livehybrid  @kiran_panchavat  thank you very much. | set the props like this, and field extractions are fine now. [netiq] SHOULD_LINEMERGE = false LINE_BREAKER = ([\r\n]+)<\d+> TIME_PREFIX = rt= TIME_FORMAT = %s%3N MAX_TIMESTAMP_LOOKAHEAD = 20 KV_MODE = auto TRUNCATE = 99999 ... View more

@livehybrid Thanks for the response. Regex is working fine, and 4 fields are extracted (log_level, request_id, component, message) Are these four fields the only ones for typo3 logs, and should this work for every typo3 log format? I did not find an official documentation on typo3 logs format. The message field contains some nested field value pairs as well. In addition, message values have multi-line events as well, so I had to adjust props.conf like this: SHOULD_LINEMERGE = true BREAK_ONLY_BEFORE = ^\w{3},\s+\d+\s+\w+\s+\d{4}\s+\d{2}:\d{2}:\d{2}\s+\+\d{4} Thanks ... View more

Thanks, Upon checking the logs, it seems MongoDB is not running on the heavy forwarder, and that would be required. ta_netskopeappforsplunk_netskope_alerts_v2.log splunklib.binding.HTTPError: HTTP 503 Service Unavailable -- KV Store initialization failed. 2: splunkd.log: ERROR KVStoreConfigurationProvider [15442 KVStoreConfigurationThread] - Failed to start mongod on first attempt reason=KVStore service will not start because kvstore process terminated  ERROR KVStoreConfigurationProvider [15442 KVStoreConfigurationThread] - Could not start mongo instance. Initialization failed.  ERROR KVStoreBulletinBoardManager [15442 KVStoreConfigurationThread] - Failed to start KV Store process. See mongod.log and splunkd.log for details. INFO  KVStoreConfigurationProvider [15442 KVStoreConfigurationThread] - Mongod service shutting down ... View more

Thanks for the response. The issue has been resolved by creating a new configuration file and moving the configurations there. Syslog-ng was not letting me modify the default conf file. ... View more

@bowesmana  Thanks for the response. We are using SVCs model. We have Splunk ES and other resource intensive apps. I am looking for a way to optimize searches as the current indexer memory utilization is 99% since last 2 months.  I used the CMC dashboard and that showing me the DM acceleration searches are the most expensive ones. ... View more