Alerting

How can I send the number of results OR all search results via a webhook triggered by an alert?

dgard
Explorer

I've tried triggering 'Once' and 'For each result', and in both cases I see only one result in the POST body send via the webhook. I've included an example below.

I was hoping for either all results, so that I could count them, or better yet a count of results. Is this possible?

Example result

{
    "app": "search",
    "owner": "emailaddress@adomain.com",
    "search_name": "Alert Name",
    "results_link": "http://a.pointless.url/that/doesn%27t/work",
    "sid": "scheduler_ZGF2aWQuZ2FyZEBsYW5kbWFyay5jby51aw__search__RMD5decc55088fa60070_at_1535711100_37050",
    "result": {
        "splunk_server_group": "",
        "_eventtype_color": "",
        "_cd": "4:960",
        "_serial": "0",
        "_sourcetype": "httpevent",
        "_bkt": "my-index-name~4~98B5C0B4-EAAF-4B7C-9775-56A8E159035D",
        "_time": "1535710877",
        "splunk_server": "splunk.adomain.com",
        "_kv": "1",
        "_indextime": "1535710877",
        "source": "my-index-name",
        "eventtype": "",
        "_raw": "This is a test, everything is really fine.",
        "host": "splunk.adomain.com",
        "index": "my-index-name",
        "_si": [
            "splunk.adomain.com",
            "my-index-name"
        ],
        "punct": "__-_",
        "sourcetype": "httpevent",
        "linecount": "1"
    }
}

My alert

  • Enabled: Yes
  • App: search
  • Permissions: Shared in App
  • Alert Type: Scheduled (Cron Schedule)
  • Trigger Condition: Number of Results is > 0
  • Actions: 2 Actions
    • Add to Triggered Alerts
    • Webhook
0 Karma
1 Solution

dgard
Explorer

All I had to do here was return a count from the search, as opposed to the full results of the search.

index="my-index-name" THIRD_PARTY_ERROR | stats count as total

View solution in original post

0 Karma

dgard
Explorer

All I had to do here was return a count from the search, as opposed to the full results of the search.

index="my-index-name" THIRD_PARTY_ERROR | stats count as total
0 Karma

sudosplunk
Motivator

Hi, what is the search you're using?

0 Karma

dgard
Explorer

I'm using a basic search, simply searching for any events with an occurrence of "THIRD_PARTY_DOWN" within a single index.

index="my-index-name" THIRD_PARTY_ERROR

May I ask, how do you think that will affect the POST body sent by a webhook?

0 Karma

sudosplunk
Motivator

I thought, you want to include total number of events (like count in the example below) in your results. Please correct me if I misunderstood your question.

{

    "result": {
        "sourcetype" : "mongod",
        "count" : "8"
    },
    "sid" : "scheduler_admin_search_W2_at_14232356_132",
    "results_link" : "http://web.example.local:8000/app/search/@go?sid=scheduler_admin_search_W2_at_14232356_132",
    "search_name" : null,
    "owner" : "admin",
    "app" : "search"
}
0 Karma

dgard
Explorer

Yep, looks like that worked. Thanks.

0 Karma

dgard
Explorer

Yes, that's the idea. I think I may have figured this out - I can append " | stats count as total" to my saved search, and that should hopefully do the job - will update when I've tested.

0 Karma

sudosplunk
Motivator

Yes. I was about to suggest the same. Append your search with stats to produce count.

0 Karma
Get Updates on the Splunk Community!

Troubleshooting the OpenTelemetry Collector

  In this tech talk, you’ll learn how to troubleshoot the OpenTelemetry collector - from checking the ...

Adoption of Infrastructure Monitoring at Splunk

  Splunk's Growth Engineering team showcases one of their first Splunk product adoption-Splunk Infrastructure ...

Modern way of developing distributed application using OTel

Recently, I had the opportunity to work on a complex microservice using Spring boot and Quarkus to develop a ...