Deployment Architecture

Splunk Metrics via Splunk Add on for *nix

Tohrment
Path Finder

So, we aren't in an environment where we can just deploy apps (a la collectd) but want to get metrics in from Linux boxes. Has anyone figured out how to accomplish this with the Splunk Add-on for *nix? I am terrible with regex(trying to change that) so I have not been able to figure out the proper transforms to get the data in a format fit for metrics index ingestion but needing to get it into our lab for future presentations on the matter(in the hopes of getting a license bump lol). Anyway, any help would be most appreciated!

1 Solution

lukeh
Contributor

Hi Tohrment,

I have recently released a new add-on to collect Linux metrics without collectd 🙂

Metrics Add-on for Infrastructure:
https://splunkbase.splunk.com/app/4856/

All feedback is welcome 🙂

L.

View solution in original post

0 Karma

lukeh
Contributor

Hi Tohrment,

I have recently released a new add-on to collect Linux metrics without collectd 🙂

Metrics Add-on for Infrastructure:
https://splunkbase.splunk.com/app/4856/

All feedback is welcome 🙂

L.

View solution in original post

0 Karma

Arijit1
Loves-to-Learn

Hello  Lukeh,

Thanks  for solution Metrics Add-on for Infrastructure.  it is  working  perfectly  for linux.  

Now I am struggling  with HPUX  ,solaris and AIX.  

It would be great  if you have solution on those OS  as  well.  

Thanks  in Advance 

Arijit Chowdhury 

0 Karma

satyenshah
Path Finder

One frustration with the Linux TA (v6.0.2) is that the output contains so much whitespace. Ingesting that data as a metric spares your license, but you still lose on storage and speed.

For example, I enabled vmstat.sh, bandwidth.sh, df.sh, and cpu.sh on one host. Those four inputs generate 64,552 bytes / hour of raw events. If I dedup the whitespace (replace \s+ with \s) then that shrinks to 36,465 bytes, which is 44% reduction.

In other words, metrics from the Linux TA are 51% whitespace. Anybody try to fix this? The whitespace originates from the awk/printf commands in the bash scripts. The commands format the output into pretty printed tables, which doesn't make sense for machine data.

0 Karma

fferozbasha
Explorer

Hi, I have recently worked on a wrapper add-on to work on top of the Splunk NIX Add-On to tap in the output and convert them in to a metric event and transport it to indexers (either via Splunk TCP in csv file format, if the forwarder is of version more than 7 OR send via HTTP Event collector, if the forwarder is version less than 7). I am still trying to do some performance test around that before I can package the same as an add-on and publish it.

Tohrment
Path Finder

That would be amazing my friend. If you need help testing let me know. We have a small lab specifically for testing new versions of splunk and add ons. Keep us posted if you would be so kind. Thanks!

0 Karma

fferozbasha
Explorer

Thanks much Tohrment. I will surely share the add-on with you for further testing. I will publish the same in splunkbase and share you the link.

0 Karma

jherring_splunk
Splunk Employee
Splunk Employee

Did you ever manage to get this setup working? I have a need to do a logs-to-metrics for *nix TA output and any shortcuts would be appreciated.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Thanks for replying, FYI 7.2 does have the logs to metrics conversion but what Ferroz is describing covers older versions!

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma

Tohrment
Path Finder

I have not found anything regarding using logs to metrics to utilize the output of the Splunk Add-on for *nix. If you have a link specifically for that and not the generalized article for Logs2Metrics I would more than be happy to look over it and attempt it.

0 Karma

gjanders
SplunkTrust
SplunkTrust

Nothing except the generalised article which is why this was just a comment and not an answer 🙂

Alerts for Splunk Admins https://splunkbase.splunk.com/app/3796/
Version Control for Splunk https://splunkbase.splunk.com/app/4355/
0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!