topic Re: Using Splunk as a monitoring tool in Monitoring Splunk

Using Splunk as a monitoring tool

Alexw1900 — Sun, 09 Apr 2017 19:55:31 GMT

Hello

We currently run an on-prem application with the following tiers;

Client - App servers - Database servers

The App servers are Windows 2012, the Database is SQL 2008 on Windows 2012.

At the moment, we have a bespoke monitoring tool that ingests logs from the App/Database servers and displays the pertinent information graphically on a web interface. We have also set up alerts for specific log entries.

Our organisations uses SPLUNK so I was thinking of a way of ingesting the logs to a SPLUNK repository and then writing a custom tool that could display the same information. This way, we can decomm one more bespoke monitoring platform.

Has anyone had any experience of this or point me in the right direction? I understand that SPLUNK has Rest API's to hook into that I'm hoping can help.

Re: Using Splunk as a monitoring tool

koshyk — Sun, 09 Apr 2017 21:33:16 GMT

Welcome to the real monitoring tool mate :). We have used Splunk as monitoring tool for 1000's of systems and have got near real time data and alerting

Well, you can use Splunk to any level of monitoring. You don't need to write any kind of custom tool in my opinion nor REST api, once you see the data. Few options for you

Get all the data for your application into Splunk (Using Splunk Universal Forwader OR syslog or DBconnect or any other methods)
Write searches and dashboards to view what you requite for monitoring
Write searches and alerts to alert you
Create reports for your team and management
If you require service modelling (impact analysis based approach) lot of free apps are available within splunk. Or there is official ITSI app which can do all wonders
if none of above is sufficient you can look into REST api or custom tool.
Any queries/help with searches post in this forum

Re: Using Splunk as a monitoring tool

Alexw1900 — Tue, 11 Apr 2017 05:56:43 GMT

That's great!

So, for example, could we monitor:

Number of TCP Connections for a server
Parse the logs to identify the number of open connections
Specific errors

...and have, say, the number of TCP connections displayed in a graph format (e.g. time vs number of connections)? And we don't need the REST API to do that, we can just write that ourselves?

Any ideas on how to get started?