Using Splunk as a monitoring tool

[Alexw1900]

Hello

We currently run an on-prem application with the following tiers;

Client - App servers - Database servers

The App servers are Windows 2012, the Database is SQL 2008 on Windows 2012.

At the moment, we have a bespoke monitoring tool that ingests logs from the App/Database servers and displays the pertinent information graphically on a web interface. We have also set up alerts for specific log entries.

Our organisations uses SPLUNK so I was thinking of a way of ingesting the logs to a SPLUNK repository and then writing a custom tool that could display the same information. This way, we can decomm one more bespoke monitoring platform.

Has anyone had any experience of this or point me in the right direction? I understand that SPLUNK has Rest API's to hook into that I'm hoping can help.

[koshyk]

Welcome to the real monitoring tool mate :). We have used Splunk as monitoring tool for 1000's of systems and have got near real time data and alerting

Well, you can use Splunk to any level of monitoring. You don't need to write any kind of custom tool in my opinion nor REST api, once you see the data. Few options for you

• Get all the data for your application into Splunk (Using Splunk Universal Forwader OR syslog or DBconnect or any other methods)
• Write searches and dashboards to view what you requite for monitoring
• Write searches and alerts to alert you
• Create reports for your team and management
• If you require service modelling (impact analysis based approach) lot of free apps are available within splunk. Or there is official ITSI app which can do all wonders
• if none of above is sufficient you can look into REST api or custom tool.
• Any queries/help with searches post in this forum

[Alexw1900]

That's great!

So, for example, could we monitor:

• Number of TCP Connections for a server
• Parse the logs to identify the number of open connections
• Specific errors

...and have, say, the number of TCP connections displayed in a graph format (e.g. time vs number of connections)? And we don't need the REST API to do that, we can just write that ourselves?

Any ideas on how to get started?