Hello Splunk Community,
I'm encountering an issue with ingesting data from a Prometheus remote_write_agent into Splunk Enterprise – this solution utilises the ‘Prometheus Metrics for Splunk and is within a Test Environment.
Problem Summary: Despite ensuring that the 'inputs.conf' file matches the configuration specifications defined in the 'inputs.conf.spec' file, the Prometheus data is not being ingested and I am receiving errors, e.g port: Not found in "btool" output (btool does not list the stanza [prometheusrw]) when viewing the inputs.conf file in the config explorer application.
Details:
Splunk Version: Splunk Enterprise 9.2 (Trial License)
Operating System: Ubuntu 22.04
Splunk Application: Prometheus Metrics for Splunk (Latest Version 1.0.1)
inputs.conf.spec
/opt/splunk/etc/apps/modinput_prometheus/README/inputs.conf.spec
(Full inputs.conf.spec - https://github.com/lukemonahan/splunk_modinput_prometheus/blob/master/modinput_prometheus/README/inp...
As seen in image, the inputs.conf.spec file states there is a port and maxClients configuration parameters.
In the inputs.conf I updated the /opt/splunk/etc/apps/modinput_prometheus/local/inputs.conf file to include the details below which meet the required formatting above:
The inputs.conf file was saved, and the Splunk Server rebooted. After rebooting the input.conf was checked to ensure the config specification where being accepted using the Config Explorer App –
These errors where received for the following configuration parameters:
However, other configuration parameters such as index, sourcetype & whitelist
Returned: 'Found in "btool" output. Exists in spec file (Stanza=[prometheusrw]) - and were accepted by Splunk.
For some unknown reason, Splunk is not recognising some of the configuration parameters above that are listed within the inputs.conf.spec file, even when formatted accordingly.
Other Information:
Prometheus remote-write-exporter details:
Splunk Index: skyline_prometheus_metrics
Any assistance is appreciated, thank you Splunk Community 🙂
