All Apps and Add-ons

ITSI episodes and alerting

Darkvader
Explorer

My question is 2 pronged:

 

1. When we receive an alert sometimes it will appear in the "Critical and High Episodes" section of the incorrect service. Why is that and how do i fix it?

2. I as an admin am able to see these alerts, but another admin is unable too. When I check his roles and capabilities, they are the same as mine so what could be the issue?

Labels (3)
0 Karma

akkoem
Explorer

1) I think you should see itsi indexes :  itsi_tracked_alerts and itsi_grouped_alerts , locate this alert and go from there. there should be a service ID in the event that you can trace to update on the way. 

2) only think I can think of is private view , perhaps you should share more context.

0 Karma

Darkvader
Explorer

Thanks! Another question. What determines how long an episode will appear under the "critical and high episodes" section? I cant seem to find out why in the docs. 

Darkvader_0-1762176709725.png

 

0 Karma

akkoem
Explorer

Regardless of it's severity , it would stay in the list as long as retention time of the it's kvstore: itsi_grouped_alerts (setting is TTL)
This could be useful when episode is closed, and one wants check what was the resolution notes from prev. incident etc..
That's being said, if you must delete episodes after certain time:

find the collections.conf where itsi_grouped_alerts is configured ( I think it was in main ITSI app but could be in one of the side apps)

[itsi_grouped_alerts]
ttl = 2592000  # 30 days in seconds


if you don't wanna deal with conf files but with pure SPL, you can schedule a daily scheduled search that will do the same work manually for you:

| inputlookup itsi_grouped_alerts
| where _time < relative_time(now(), "-30d")
| outputlookup itsi_grouped_alerts append=false

 please note this is un-reversible. 

Finally, setting a retention time on indexes.conf for notable index would do the final work for episodes. 

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Thanks for the Memories: .conf26 Took Learning to New Heights

Thank you, Splunk Community, for making .conf26 in Denver one for the books. From packed Splunk University ...

Best Practices: Splunk auto adjust pipeline queue

When you enable autoAdjustQueue in Splunk, maxSize should be understood as the queue size Splunk starts with ...

Splunk Auto Ingestion Parallel Pipeline Scaling

Why this feature matters Many Splunk environments experience ingestion pressure long before the host is fully ...