Occasionally, alerts will fire that seem to get stuck and never make it into the Incident Posture dashboard. When I check the Incident Status KPI dashboard, they show up in the Long Runners but searching for the incident id yields no results. Please advise. Thanks!
-Josh
I suspect, that some large events created as "index=alerts sourcetype=alert_metadata" may get truncated.
There's a bugfixes version of TA-alert_manager you could try out:
https://github.com/alertmanager/TA-alert_manager/tree/develop
Download as a zip file, and rename the directory to TA-alert_manager
Thanks, I'll give that a try and report back.