Archive

Setting up splunk monitors

New Member

At my job whenever they set up a Splunk forwarder they add only one monitor. "/var/logs". Does anybody do it this way?

Shouldn't we adding monitors with stanzas on the /SPLUNK_HOME/etc/system/local/intputs.conf ?

We have forwarders installed on 29 servers. Our licence usage is currently at 8GB. Does this sound like its too much for the amount of servers?

Thanks in Advance

0 Karma

Ultra Champion

if this is as you say, the chances are high that your ingesting duplicate data (such as when your log files role) and you likely have very few sourcetypes.

Whilst obviously this will (and does) work, its not a very sensible way to use Splunk.

In terms of estimating the licence usage, its difficult to say without knowing what sort of logs your collecting but 8GB/day for 30 servers seems like a lot unless they are quite busy.

0 Karma

New Member

Yeah that's what I was thinking. All of the instances have the Splunk Add-on for Unix and Linux. Some alerts are just set up to check if a service is running. I don't think we even need to add that monitor directory in order for those alerts to work.

0 Karma

Ultra Champion

The TA for nix comes configured to collect a number of common logs from *nix systems, so its possible that's how your environment has been configured. If so you probably have sourcetype=messages or sourcetype=dmesg.

If this is the case then it may not be as you fear.
If however, all of your data is in one sourcetype I shall weep for you. 🙂

0 Karma

New Member

what do you mean by all of my data is once sourcetype?... and I have a feeling it is lol..

So lets say I am trying to monitor only the tomcat service and create a query with ps source. I should go into /etc/system/local/inputs.conf and add:

[monitor:/opt/tomcat/logs/catalina.out]

then this should work and it shouldn't return any results when the service is down correct?

host="server1" source=ps tomcat| stats latest(_time) as latest by host
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!