What is the best way to setup Alerts for App Agent Status and Machine Agent Status?
When the Agent is turned off, the Agent Status stops reporting any values. So what, then, is the best approach for setting up a Health Rule?
Solution 1 - Machine Agent's Metric
The machine agent posts a metric you can use:
Application Infrastructure Performance|FrontEnd|Agent|Machine|Availability
Solution 2 - App Agent Status
You can also use the App Agent Status for app agent availability.
But note that, in addition to the app pool recycles that take place by default every 29 hours, there is also an app pool idle time out that occurs if there’s no traffic on the app after 20 minutes. If that idle occurs, then the agent will be unloaded, will no longer report and trigger this alert, and look like a false alert (App|Availability = 0).
You can address this by turning that setting off by setting the timeout value to 0.

Read about this here under the section Idle Time Out.