So I am setting up Splunk alerting. I want to devise an alert such that it monitors hosts. Specifically if nothing is being written on the host disk. This would run on if nothing is being written on the host HDD for X number of days, then alert else not. Is there a way to monitor this aspect in Splunk? Any help is appreciated.
Are you planning on monitoring Logical Disc usage of your host machine? or Whether you need to check whether specific forwarding/event collection from your host machine to Splunk is working or not?
Answer to your first scenario depends on which type of system you are monitoring. For example if you are monitoring Windows OS based host machine, you need to collect Windows performance counter for Logical or Physical disc i.e. Perfmon:LogicalDisk or Perfmon:PhysicalDisk based on your need and setup alert on specific counter like Disk writes or free space compared over time.
Refer to the following documentation for Windows Performance Counter monitoring: http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorWindowsperformance
For second scenario, if you are already forwarding events from host machine/s to Splunk, then you can use tstats to perform fast statistical check on indexed metadata to setup alert.
| tstats count as EventCount latest(_time) as Time where host=<YourHostName> by sourcetype | eval Time=strftime(Time,"%c")
Based on your needs you can switch sourcetype and host between where and by clauses. Refer to tstats documentation for details: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Tstats
Are you planning on monitoring Logical Disc usage of your host machine? or Whether you need to check whether specific forwarding/event collection from your host machine to Splunk is working or not?
Answer to your first scenario depends on which type of system you are monitoring. For example if you are monitoring Windows OS based host machine, you need to collect Windows performance counter for Logical or Physical disc i.e. Perfmon:LogicalDisk or Perfmon:PhysicalDisk based on your need and setup alert on specific counter like Disk writes or free space compared over time.
Refer to the following documentation for Windows Performance Counter monitoring: http://docs.splunk.com/Documentation/Splunk/latest/Data/MonitorWindowsperformance
For second scenario, if you are already forwarding events from host machine/s to Splunk, then you can use tstats to perform fast statistical check on indexed metadata to setup alert.
| tstats count as EventCount latest(_time) as Time where host=<YourHostName> by sourcetype | eval Time=strftime(Time,"%c")
Based on your needs you can switch sourcetype and host between where and by clauses. Refer to tstats documentation for details: https://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Tstats