Hello Everyone,
Recently I have installed Splunk db connect app (3.16.0) in my Splunk heavy forwarder (9.1.1). As per the documentation I have installed jre and installed the my sql add on. And created Identities, connections and inputs. But when I check for the data it is not getting ingested. So I enabled the debug mode and checked the logs and got the hec token error. But the hec token is configured in inputs.conf file and same I could see in the Splunk web gui under Data inputs -> HTTP event collector.
could you please help if anyone faced this error before?
Error:
ERROR org.easybatch.core.job.BatchJob - Unable to write records java.io.IOException: There are no Http Event Collectors available at this time.
ERROR c.s.d.s.dbinput.recordwriter.CheckpointUpdater - action=skip_checkpoint_update_batch_writing_failed java.io.IOException: There are no Http Event Collectors available at this time.
ERROR c.s.d.s.task.listeners.RecordWriterMetricsListener - action=unable_to_write_batch java.io.IOException: There are no Http Event Collectors available at this time.
It sounds like some sort of setting related to connection issue:
A few thinsg to check:
Is there a firewall between your DB connect server and the HEC server?
Ensure the port(s) are availble
Ensure on Splunk HEC server, you have global settings enabled:
Click Settings > Data Inputs.
Click HTTP Event Collector.
Click Global Settings.
In the All Tokens toggle button, select Enabled.
Some other aspects to check and troubleshoot:
#Check if the Hec collector is healthy
curl -k -X GET -u admin:mypassword https://MY_Splunk_HEC_SERVER:8088/services/collector/health/1.0
#Check if HEC stanzas with config are configured
/opt/splunk/bin/splunk http-event-collector list -uri https://MY_Splunk_HEC_SERVER:8089
#Check the settings using btool
/opt/splunk/bin/splunk cmd btool inputs list --debug http