monitor to check whether the logging (receiving) is working or not !!!

[basilboon]

Hi Splunk Team,

First of all you got a great app !! Thanks for that !!

My master Splunk has setup correctly and forwards logs to another active splunk. Yesterday when checked the log receiving has been stopped because of the disk space in the server. It got fixed after increasing the disk space.

Now the thing is, we are trying to setup a monitor using our tool (ICINGA) just to check whether the log receiving is up to date.

Is there any command to show whether the logs are up to date ?

Let me know if you need more information.

Regards,
Basil

[basilboon]

Hi Daniels,

Thanks for your reply.

The exact thing I want is to write a shell script (bash) to monitor whether the logging is working properly. The script will run in every five min and get the data (some how via shell) and send the mail to a distribution list, if only the logging is not working for the past one hour or so.

Just wanna know if there is any commands to identify this from back end (server console).

Regards,
Basil

[sdaniels]

Check out this previous Splunkbase answer. You can create an alert based on that search. You'll just need to adjust the time to be 'age > somenumber' in seconds. The example below is checking to see if there are any hosts that haven't sent events in the last two days. If the search comes up empty it means you are ok. If you get a values you can alert on them to let you know which hosts might be having issues sending data to Splunk.

http://splunk-base.splunk.com/answers/3181/how-do-i-alert-when-a-host-stops-sending-data

| metadata index=main type=hosts | eval age = now()-lastTime | where age > (2*86400) | sort age d | convert ctime(lastTime) | fields age,host,lastTime

[basilboon]

Hi Daniels,

Thanks for your reply.

The exact thing I want is to write a shell script (bash) to monitor whether the logging is working properly. The script will run in every five min and get the data (some how via shell) and send the mail to a distribution list, if only the logging is not working for the past one hour or so.

Just wanna know if there is any commands to identify this from back end (server console).

Regards,
Basil