All Apps and Add-ons
Highlighted

How to get alerts in alertmanager to show as incidents?

New Member

Hi All,

I've configured two alerts from the search app in Splunk and did 'save as' from the search bar. These do appear the 'alerts' tab within Alert Manager, but nothing shows up 'incident posture'.

In permission settings for the alert, I've set it to read/write for all Apps, trigger in real-time and a trigger action to call alert manager.

Nothing shows up in 'recent incidents' or in the color coded incident counters in 'incident posture'.

In settings > Incident settings:

" _Key " column is populated, alert column is populated with a name, however 'category, subcategory, tags' are unknown/untagged.

What am I missing please?

Thanks

0 Karma
Highlighted

Re: How to get alerts in alertmanager to show as incidents?

Explorer

I can't say why this is happening, but the 'category, subcategory, tags' are optional fields.

First I would make sure your user you are looking at the dash has alert manager permissions.

Second I like to use the "Add to Triggered Alerts" option from alert actions along with the "Alert Manager" option. This way you can verify your alert is working as expected with the splunk alerting subsystem outside of alert manager.

Once you have alerts showing in "Triggered Alerts" you should see them in the dashboard context.

you can also check the index you assigned at install for any incidents, and the kvstore too with

 | `all_alerts`

to see if there is anything there

0 Karma
Highlighted

Re: How to get alerts in alertmanager to show as incidents?

New Member

The user for alert manager is the admin account created at install, this was in there by default.

I've used the 'add to triggered alerts' option, and can see the output in Alert Manager > Alerts > Configured Alert. It loads a menu with 'trigger history' and in the actions column you can do 'view history'. So I can see there is some output recorded for those alerts setup.

Just can't work out why all the incident numbers are 0, even with the time range set to way back.

0 Karma
Highlighted

Re: How to get alerts in alertmanager to show as incidents?

Explorer

I've just installed alert manager and I seem to be having the same problem. The alerts are triggering and I can see the incidents in the KV store, but nothing on the dashboard. Did you find a solution?

0 Karma
Highlighted

Re: How to get alerts in alertmanager to show as incidents?

New Member

Here are a couple screenshots to describe what the issue is

alt text

alt text

0 Karma