Getting Data In

Cannot Ingest Prometheus Data: inputs.conf - recieving errors: btool does not list the stanza [prometheusrw])

Network007
Loves-to-Learn

Hello Splunk Community, 

I'm encountering an issue with ingesting data from a Prometheus remote_write_agent into Splunk Enterprise – this solution utilises the ‘Prometheus Metrics for Splunk and is within a Test Environment.

Problem Summary: Despite ensuring that the 'inputs.conf' file matches the configuration specifications defined in the 'inputs.conf.spec' file, the Prometheus data is not being ingested and I am receiving errors, e.g port: Not found in "btool" output (btool does not list the stanza [prometheusrw]) when viewing the inputs.conf file in the config explorer application.

Details:

Splunk Version: Splunk Enterprise 9.2 (Trial License)

Operating System: Ubuntu 22.04

Splunk Application: Prometheus Metrics for Splunk (Latest Version 1.0.1)

 

inputs.conf.spec

 /opt/splunk/etc/apps/modinput_prometheus/README/inputs.conf.spec

(Full inputs.conf.spec - https://github.com/lukemonahan/splunk_modinput_prometheus/blob/master/modinput_prometheus/README/inp...

As seen in image, the inputs.conf.spec file states there is a port  and maxClients configuration parameters.

Network007_0-1718958600783.png

In the inputs.conf I updated the  /opt/splunk/etc/apps/modinput_prometheus/local/inputs.conf file to include the details below which meet the required formatting above:

Network007_1-1718958660305.png

The inputs.conf file was saved, and the Splunk Server rebooted. After rebooting the input.conf was checked to ensure the config specification where being accepted using the Config Explorer App – 

These errors where received for the following configuration parameters:

Network007_2-1718958697322.png

Network007_3-1718958712578.pngNetwork007_4-1718958722647.png

However, other configuration parameters such as index, sourcetype whitelist 
Returned: 'Found in "btool" output. Exists in spec file (Stanza=[prometheusrw]) - and were accepted by Splunk.

Network007_6-1718958848185.png

For some unknown reason, Splunk is not recognising some of the configuration parameters above that are listed within the inputs.conf.spec file, even when formatted accordingly.

 

Other Information:

Prometheus remote-write-exporter details:

Network007_7-1718958974448.png

Splunk Index: skyline_prometheus_metrics

Network007_8-1718959016503.png

Any assistance is appreciated, thank you Splunk Community 🙂

 

 

 

 

 

Labels (2)
0 Karma
Get Updates on the Splunk Community!

Optimize Cloud Monitoring

  TECH TALKS Optimize Cloud Monitoring Tuesday, August 13, 2024  |  11:00AM–12:00PM PST   Register to ...

What's New in Splunk Cloud Platform 9.2.2403?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.2.2403! Analysts can ...

Stay Connected: Your Guide to July and August Tech Talks, Office Hours, and Webinars!

Dive into our sizzling summer lineup for July and August Community Office Hours and Tech Talks. Scroll down to ...