I've found an interesting specific case where there are two callRecord with the same id, both with version=1, but one is a peerToPeer call and the other is a groupCall. I think there are multiple callRecords because the initial peerToPeer call had a third participant added, escalating it to a groupCall. This could also explain some apparent duplication. ... View more

Looking at the webhook events in more detail reveals my first wrong assumption: a single call can produce multiple webhook events, with one of two changeTypes: 'created' or 'updated'. The longer the call goes on for, the more changeType:updated events are pushed to the webhook. However, looking at callRecord events with a matching id it gets stranger. I can see 15 webhook (one 'created' and 14 'updated') events with the same id today with Splunk _time values between 10:15 and 12:15. But there are (only) 8 matching callRecord events all with the same Splunk _time value of 07:30, startDateTime of 07:30 and endDateTime of 09:53, each with a different 'version' of  1, 2, 3, 4, 5, 8, 12 or 15, and an incrementing lastDateTimeModified value (between 10:14 and 12:12) I thought the _time value in a splunk event showed when it was created. How can these callRecord events all have been created at 07:30, for a call that was in place between 07:30 and 09:53, and have webhook events between 10:15 and 12:15? ... View more

Thanks for the replies, both of you! I was, indeed, close. I didn't realise you could continue to process the output of a stats command further, so @richgalloway's solution is pretty obvious when you grok that. @ITWhisperer, I'd not used eventstats before, so this was a good one to learn. Your code outputs a line for each activeNumber per month, though. So where Rich's outputs a line for every activeNumber that is active in all the months, yours output a line per month per activeNumber. So the overall number of activeNumbers, multiplied by the number of months searched for. Both answers are great, though, I understand how they work and why they give different values. Thanks both again! ... View more

mvappend is interesting, but doesn't seem to do what I want. I'm seeing numbers returned that I know have received a call in the search period. I'm not sure how searching a multi-field result value a list of single-field values works, particularly with the NOT involved. If a DN in the inputlookup file is present as one or more of callingPartyNumber, originalCalledPartyNumber or finalCalledPartyNumber it shouldn't be returned as a result. My original query was returning every number that didn't appear as any one (or more) of the fields, I'm not sure what logic is being applied to the mvappend version, as that produces a different result. ... View more

Well, out of curiosity I went back and set up a separate webhook on a different port and it's still not working properly, I'm getting 404 errors for nonexistent call records. Whenever I (re)start the subscription for the second tenancy I can see the *wrong* webhook being restarted (they're running on different ports). I'm not convinced that, even with an additional webhook configured, the Graph API is being told the correct webhook to use. Sadly I think there's something missing in the code for this to allow multiple tenancies to run on one node. FWIW, I'm running this on a heavy forwarder with separate indexer and search nodes. ... View more

Having just set this up for my system (and the steps being fresh in my memory): did the client secret for your Azure AD app expire? ... View more

Okay, I got this up and running for our test tenancy first with no problems. Generic webhook, plus subscription and call record inputs, and account unique to that tenancy and it's Azure AD app. I then created a new Azure AD app in the live tenancy, followed by a subscription and call record input on Splunk with the live tenancy's tenantID, and a new account with the live tenancy's clientID and secret. The subscription and call record inputs both point at the previous generic webhook. Call records are being ingested successfully from both tenancies into the same index, and are identifiable using the 'source' field in the records as I gave them uniquely identifiable names relating to the tenancy name. All good so far, but the wrinkle seems to be that every call record header that gets pushed to the webhook is passed onto *both* call record inputs to be requested from graph.microsoft.com. One always fails, as the call either took place on live or testing but not both, and returns a 404. It seems that the intelligence isn't there for the webhook to extract a tenantID from the call record header and pass it on to the relevant call record input. Disabling the call record input for the testing tenancy (where there are very few calls going on) bears this out. This works for my needs, I can turn the logging on and off for the testing tenancy as and when I need it, but if you're hoping to capture logs for multiple live tenancies I'd be inclined to set up a unique webhook for each one. ... View more

I was wondering the same thing as I'm about to set this up for our testing tenancy, and then our live one. It would be handy to have all data in one index via one "add on" as we can obviously differentiate between tenancies using tenantID in the data. I'll give it a try and report back if no one else replies before me. ... View more

Brilliant! It was missing a bracket but did the trick once I popped it back in 🙂 The most important part seems to be eval callParticipents=mvsort(split(callingPartyNumber."#".finalCalledPartyNumber,"#")) ...but I can't quite figure it out, could you explain it, please? I get that we're creating a new multivalue field to work with called callParticipents for each event in the timeframe. I guess it doesn't matter which order the values go into the field A/B is the same as B/A for the purposes of a multivalue field, right? I just can't grok how callParticipents is built from that line edit Never mind, I sat down and read the docs on split and the mv commands and I get it now. Thanks so much! ... View more