topic Re: How to query Server Stats in Splunk Search

How to query Server Stats

gherkin — Thu, 04 Nov 2021 21:29:26 GMT

Good afternoon

i'm wondering if I may be able to get a bit of help with this one as I'm struggling on trying to achieve what I want. I would like to query my 3 servers about their hardware status such as how much space is on the HDD etc etc however i'm really struggling to get my head around how to go about achieving this.

I've seen a few posts on here which refer to making changes to the input.conf file by adding perfmon but firstly i'm not 100% sure on which input.conf i should be doing this on (i'm presuming the forwarder) if this is at all the case, and secondly i'm not sure where and how this information in gleamed from. If anyone would be able to point my in the right direction to a resource that is a step by step guide (or there abouts) i would be very grateful.

TIA

Re: How to query Server Stats

isoutamo — Thu, 04 Nov 2021 21:37:12 GMT

if you are using linux then you should use this https://docs.splunk.com/Documentation/UnixAddOn/6.0.0/User/AbouttheSplunkAdd-onforUnixandLinux and follow that configuration instructions https://docs.splunk.com/Documentation/UnixAddOn/6.0.0/User/Enabledataandscriptedinputs

If you are using HF then you could configure that via GUI otherwise you must use those configuration files.

r. Ismo

Re: How to query Server Stats

gherkin — Thu, 04 Nov 2021 21:42:21 GMT

Hi @isoutamo

I'm using windows machines i'm afraid, I'm also using a UniversalForwarder - I've just read that perhaps I need to enable the introspective_generator_addon. Is that correct?

Re: How to query Server Stats

isoutamo — Thu, 04 Nov 2021 21:46:56 GMT

You can try this one https://docs.splunk.com/Documentation/AddOns/released/Windows/AbouttheSplunkAdd-onforWindows

Just install it on your windows machines which are running UF. Do configuration via conf-files or install it first on some HF/your test server and then take conf-files there and install then those (without host name) to UFs.

Re: How to query Server Stats

Roy_9 — Fri, 05 Nov 2021 03:41:11 GMT

Hello,

Best solution is to install the splunk add on for windows available on splunk base on the windows server where splunk agent is installed.

this package has got all system level and perfmon metric collection inputs

just enable those, it will be a lot easier for your situation.

Re: How to query Server Stats

gherkin — Wed, 10 Nov 2021 04:13:10 GMT

Good evening @isoutamo

apologies for the delay in response, i was pulled away from the office the last couple of days.

So, i believe i've got it installed correctly, utilised the deployment server and edited the inputs.conf file enabling the sections that I want to monitor - checking the servers they seemed to have pulled down the files correctly.

I've also created an index (client_monitoring) and used:

index = client_monitoring

in the inputs.conf file. However if I try and do a search on that index I get No results found.

If I look at the Index's management page no data appears to be coming in as the Event Count is 0.

Apologies if this is all quite straight forward, I'm trying to teach myself this as I go along lol

Re: How to query Server Stats

isoutamo — Wed, 10 Nov 2021 08:59:04 GMT

Have you rebooted the UF service after installing and (every) configuration changes?

Have you gotten those internal logs from that UF server or is those application logs only which are missing?

Re: How to query Server Stats

gherkin — Wed, 10 Nov 2021 17:49:26 GMT

good morning @isoutamo so I did restart the UF on one server (i was just testing it at the time to see if it works).

Apologies, i don't fully understand you logs sentence.

Re: How to query Server Stats

isoutamo — Wed, 10 Nov 2021 19:27:51 GMT

Internal logs are UF’s own logs which told how it works. Application logs are e.g. windows, web servers or other logs which are generated by some application.

Re: How to query Server Stats

gherkin — Fri, 12 Nov 2021 00:50:58 GMT

Ok cool, gotcha. So restarted the UF again today just to make sure and everything has kicked in, so either I restarted it to soon last time (ie before the conf file had come down) or it didn't necessarily start correctly.

Thank you once again, onto the my next research topic.....Dashboards 🙂