Getting Data In

Splunk 7.0.0: How to get metrics in from collectd

sureshr7
Explorer

Hi,

How to add a tag(region) to a collectd based metric from a host? For example if we have 2 regions (us-east,us-west), how to add this data from collectd to the Splunk metrics index?

Thanks
Suresh

sureshr7
Explorer

Using the below code, write_http will add custom tag "meta":{"foo":"bar"}" to all json data. Now the challenge is recognizing this as dimension in splunk.


PostCacheChain "PostCache"

<Target "set">
  MetaData "foo" "bar"
</Target>

Target "write"

0 Karma

lukeh
Contributor

Thanks, I'll have a crack and get back to you.

0 Karma

theouhuios
Motivator

Any luck with this? Is there a way to add Tags via collectd and get splunk to understand them as dimensions?

0 Karma

sureshr7
Explorer

I was able to add a custom meta tag using PostCacheChain from collectd. But still not able to read it via splunk.

0 Karma

lukeh
Contributor

I would be happy to test this for you, could you please provide an example from collectd.conf ?
FYI: I am the author of Analytics for Linux which uses metrics from collectd:
https://splunkbase.splunk.com/app/3777/

0 Karma

thaggie_splunk
Splunk Employee
Splunk Employee

You should be able to add a region field as you push the measurements to splunk.
From: http://docs.splunk.com/Documentation/Splunk/7.0.0/Metrics/GetMetricsInCollectd

curl -k https://localhost:8088/services/collector/raw?sourcetype=collectd_http   \
-H "Authorization: Splunk <HEC_token>"                                      \
-d '[{"values":[164.9196798931339196],"dstypes":["derive"],"dsnames":["value"],"time":1505356687.894,"interval":10.000,"host":"collectd","plugin":"protocols","plugin_instance":"IpExt","type":"protocol_counter","type_instance":"InOctets"}]'

So each message is like:

{
  "values": [164.9196798931339196],
  "dstypes": ["derive"],
  "dsnames": ["value"],
  "time": 1505356687.894,
  "interval": 10.000,
  "host": "collectd",
  "plugin": "protocols",
  "plugin_instance": "IpExt",
  "type": "protocol_counter",
  "type_instance": "InOctets"
}

You should be able to add:

"region": "us-east"

lukeh
Contributor

that will work for custom metrics, but not for native collectd metrics as the Collectd API does not have a concept of tags. It looks like the Collectd community is working towards a solution – but it still looks like a bandaid attempting to work around the fact that Collectd’s core has no concept of tags:
https://github.com/collectd/collectd/pull/1655

0 Karma
Get Updates on the Splunk Community!

This Week's Community Digest - Splunk Community Happenings [9.26.22]

Get the latest news and updates from the Splunk Community here! Upcoming User Group Events! &#x1f44f; Check ...

BSides Splunk 2022 - The Call for Papers is now Open!

TLDR; Main Site: https://bsidessplunk.com CFP Site: https://bsidessplunk.com/cfp CFP Opens: December 15th, ...

Sending Metrics to Splunk Enterprise With the OpenTelemetry Collector

This blog post is part of an ongoing series on OpenTelemetry. The OpenTelemetry project is the second largest ...