Getting Data In

What's the best way to get Windows Perfmon data into a Metrics Index?

Tohrment
Path Finder

The question pretty much sums it up.

I am wanting to get PerfMon data into a Metrics index and have been banging my head against it for about a week now. So far, I have been unsuccessful in my endeavor.

I was attempting to get it directly from the universal forwarder, but apparently the data is not formatted properly. I am pretty new to the wide world of Splunk, and have been reading anything I can find even slightly related to this, but there doesn't seem to be much out there as of now.

If there is a better way to do it? I am all ears(or eyes in this case).

Thanks Splunkers.

0 Karma
1 Solution

Tohrment
Path Finder

Ok, so here is where we went wrong and why we weren't getting any metrics in.

1.) INSTALL THE ADD-ON to ALL layers to ensure the proper transforms occur. I was unaware it wasn't on our HF or indexers(we have a guy that runs all that side of things).

2.) Following gjanders link confirmed that I had set up the proper local/inputs.conf stanzas but due to #1 not being done we were not getting data.
Collect perfmon data and wmi:uptime data in metric index

3.) Make sure the entire stack is up to date(at the time of this writing it is 7.2)

Thank you gjanders for pointing me to that link. Literally showed me the last bolt I was missing to make this thing work!

View solution in original post

Tohrment
Path Finder

Ok, so here is where we went wrong and why we weren't getting any metrics in.

1.) INSTALL THE ADD-ON to ALL layers to ensure the proper transforms occur. I was unaware it wasn't on our HF or indexers(we have a guy that runs all that side of things).

2.) Following gjanders link confirmed that I had set up the proper local/inputs.conf stanzas but due to #1 not being done we were not getting data.
Collect perfmon data and wmi:uptime data in metric index

3.) Make sure the entire stack is up to date(at the time of this writing it is 7.2)

Thank you gjanders for pointing me to that link. Literally showed me the last bolt I was missing to make this thing work!

View solution in original post

sloshburch
Ultra Champion
0 Karma

gjanders
SplunkTrust
SplunkTrust

Can you confirm your following Configure the Splunk Add-on for Windows, in particular Collect perfmon data and wmi:uptime data in metric index

And as per Compatibility between forwarders and Splunk Enterprise indexers you are using the 7.x or above universal forwarder and indexer?

Tohrment
Path Finder

I have it setup precisely how that is set up and it is giving me the following error

Search peer 1 has the following message: Index Processor: Metric name is missing for source=Process, sourcetype=Process, host=servername, index=winmetrics. Metric event data without metric name is invalid and would not be indexed. Ensure the input metric data is not malformed.

I have it set up exactly how the first link you gave me is set up. Do you have this working?

0 Karma

gjanders
SplunkTrust
SplunkTrust

Converted to tn answer so you can either accept this one or accept your answer.

Glad I could help!

0 Karma

sudosplunk
Motivator

Hi there, are you using the app built by splunk1 to collect perfmon data from windows? Also, have a look at splunk docs on perfmon2 to get some more insights. HTH!

0 Karma

Tohrment
Path Finder

We use the Splunk add-on for Windows and it has the perfmon stanzas in the inputs.conf. Those however do not provide the data in a way that the Metrics index will accept. What I am aiming to do is create the stack in the Metrics Workspace to be able to dig down through related objects when an issue arises. I have been unable to find anything other than "Getting Data in from other Sources" and it talks about HEC and metrics-csv inputs but no links to best practices on creating the CSVs for ingest to the Metrics index.

Basically, what the question is based around is what is the best way to create the csv or transform the perfmon data into a csv for ingest into the metrics index. If that is not the best way to go about it than what is the suggested path to get that data in a format that a metrics index will accept?

Hopefully that clears up what this question is really about.

0 Karma

sudosplunk
Motivator

Ah, got it. Thanks for the clarification. Have a look at "maciep's" answer in the below link and see if it something that will meet your requirement. If you've already tried that approach, then pls let me know what are the issues you're facing.

https://answers.splunk.com/answers/607304/sending-perfmon-data-to-metrics-index.html

0 Karma

Tohrment
Path Finder

so, that did not work. Testing some other transforms in the hopes of stumbling my way into an answer lol

0 Karma

ntankersley_spl
Splunk Employee
Splunk Employee

Add-on for Windows Infrastructure 5.0.0 and later supports metrics transformations. You can also use Windows TA 4.8.4 with Splunk App & Add-on for Infrastructure to accomplish the same thing.

0 Karma

Tohrment
Path Finder

I have looked at that but without anyone confirming whether it worked or not made me skeptical. I will talk with my props\transforms guy and see if he thinks it would work. Thanks for all the help and I will report back if it works.

0 Karma
.conf21 CFS Extended through 5/20!

Don't miss your chance
to share your Splunk
wisdom in-person or
virtually at .conf21!

Call for Speakers has
been extended through
Thursday, 5/20!