Alerting

Journey monitoring and alerting

rchhe
Engager

We're currently using splunk for  traditional dashboards, monitoring and alerting and it means that we're now very effective at identifying and addressing errors and exceptions in our apps when they occur.

We're now looking to build more sophisticated monitoring that looks for issues across journeys that users complete in our app and helps us to identify more subtle issues that might not involve errors or exceptions.

For example a simplified version of our sign up process looks something like

1. Receive sign up request

2. Create account record

3. Write account created message to queue

4. Read account created from message queue and send welcome email

Each of the steps are logged in splunk and there's a common correlation id logged across each of the steps. Any exceptions that occur trigger alerts so that's all good.

On occasion we might have an issue where messages at step 4 stop being read from the queue and the welcome emails are not sent but nothings throwing exceptions and it's not obvious anythings wrong until a customer contacts us to flag the missing sign up email.

At that point we can query splunk and see that steps 1, 2, and 3 completed successfully, but there's no logs for step 4 which indicates an issue that need investigating.

We'd like to be able to automated the process for checking that all of the expected steps in a given journey are completed and alert when steps are missed. Is there a way we can achieve this with splunk?

I've seen a question around visualising order journeys (https://community.splunk.com/t5/Dashboards-Visualizations/How-to-Visualize-Order-Journey-through-spl... which sounds in the same ballpark and refers to Splunk Business Flow, but the docs indicate it's no longer available to purchase... Are there other out of the box or paid options?

Labels (1)
0 Karma

richgalloway
SplunkTrust
SplunkTrust

You may not need anything fancy like Business Flow.  Perhaps all you need is a simple query that searches for all 4 events and correlates them by ID.  Throw an alert if the query sees a set of less than 4 events in an appropriate amount of time.

---
If this reply helps you, Karma would be appreciated.
Get Updates on the Splunk Community!

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud

Introduction to Splunk Observability Cloud - Building a Resilient Hybrid Cloud  In today’s fast-paced digital ...

Observability protocols to know about

Observability protocols define the specifications or formats for collecting, encoding, transporting, and ...

Take Your Breath Away with Splunk Risk-Based Alerting (RBA)

WATCH NOW!The Splunk Guide to Risk-Based Alerting is here to empower your SOC like never before. Join Haylee ...