Followed you Prometheus app setup.
On Heavy forwarder its up and listening on 8098
on Prometheus side seems ok to
However in Prometheus Im been thrown errors on "Unsupported Media Type"
msg="non-recoverable error" count=9 exemplarCount=0 err="server returned HTTP status 415 Unsupported Media Type: <!doctype html><html><head><meta http-equiv=\"content-type\" content=\"text/html; charset=UTF-8\"><title>415 Unsupported Media Type</title></head><body><h1>Unsupported Media Type</h1><p>The requested URL does not support the media type sent.</p></body></html>"
IN prometheus.yml in remote_writes
- url: "https://myhost.mydomain.com:8089"
authorization:
credentials: "ABC123"
tls_config:
insecure_skip_verify: true
write_relabel_configs:
- source_labels: [__name__]
regex: expensive.*
action: drop
in Splunk HF:
[prometheusrw]
port = 8098
maxClients = 10
disabled = 0
[prometheusrw://http_status]
bearerToken = ABC123
index = prometheus
whitelist = *
sourcetype = prometheus:metric
disabled = 0
Help Appreciated
@luke_monahan effectively a typo . Thanks again, works like a charm
Your Prometheus config is sending the data to port 8089 -- the main Splunkd port, rather than the prometheusrw port.
The prometheusrw input opens up a new port (8098) that can handle the content-type that Prometheus will send. You need to update your Prometheus configuration to use port 8098 and (in this case) use http and not https.