topic Regex sought for consecutive-multi-line-search-joined-on-common-id display in Splunk Search

Regex sought for consecutive-multi-line-search-joined-on-common-id display

achetreanu — Mon, 24 Mar 2014 21:41:16 GMT

This question is related to http://answers.splunk.com/answers/127725/consecutive-multi-line-search-joined-on-common-id

From my logs, I need to extract this pattern (by unique ID, these 3 lines need to be consecutive):

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPromptswelcome.vox

AAA-PROD-IVR1 DL 01:46:34.407|FYI|69/12345678 USR_PLAYPROMPT in Connected

AAA-PROD-IVR1 DL 01:46:38.167|FYI|69/12345678 GCEV_DISCONNECTED in Connected

The first line can look like this:

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPromptswelcome.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 SomeCallFlow c:MyPrompts\C1\e123.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 SpecialCallFlow \\somereallyspecialprompts\why_here\C1\e123.vox

It was easy(er) to come up with a regex to extract and qualify prompts (i.e. IvrPromptswelcome.vox vs e123.vox etc)

This is my Splunk Filter for Prompts:

host=*prod-ivr* | rex "FYI|(?<call_id_ivr>\S+)" | transaction call_id_ivr maxevents=3 startswith=vox endswith=GCEV_DISCONNECTED | rex field=_raw "(?<Prompt>(?i)[\w\s\(\)]*\.vox)" | top 40 Prompt

I can't figure out how to extract and qualify the "CallFlow" and see which one is more prevalent within this pattern.

What should be my Splunk Filter for CallFlows ?

Thank you!

A.C.

Re: Regex sought for consecutive-multi-line-search-joined-on-common-id display

kristian_kolb — Tue, 25 Mar 2014 00:32:58 GMT

What do you mean by 'CallFlow'? The string that includes the substring CallFlow, i.e. VoiceCallFlow, SpecialCallFlow etc? Or is it whatever comes between the call_id_ivr and the path to some .vox file, i.e. it does not necessarily contain the string 'CallFlow' at all? Is it only interesting to extract this piece of information for the first line of the three that make up the transaction?

Here is an example (second rex) that will extract what comes between the call_id_ivr and something that ends in .vox, so it will only be extracted for the first event in the transaction, as it's the only one containing ".vox";

host=*prod-ivr* | rex "FYI|(?<call_id_ivr>\S+)\" | rex "\s(?<CallFlow>\S+)\s\S+\.vox"| transaction call_id_ivr maxevents=3 startswith=vox endswith=GCEV_DISCONNECTED | rex field=_raw "(?<Prompt>(?i)[\w\s\(\)]*\.vox)" | top 40 Prompt

Adjust your search as needed.

Re: Regex sought for consecutive-multi-line-search-joined-on-common-id display

achetreanu — Tue, 25 Mar 2014 22:17:11 GMT

Thank you Kristian! Completely missed out that I can use "CallFlow".
This was what worked:

host=*prod-ivr* | rex "FYI|(?<call_id_ivr>\S+)" | transaction call_id_ivr maxevents=3 startswith=vox endswith=GCEV_DISCONNECTED   | rex field=_raw "\s(?<State>\w+CallFlow)" | top 10 State

I wasn't clear in my question, I was looking to extract top Call Flows during which the user hangs up. So I was looking to extract "CallFlow", completely missing the convenient naming (i.e. each state has the pattern "CallFlow").

I will abuse your kindness and throw another problem at you 🙂

Next, I'd like to have some sort of average 'size' of my prompts. Each prompt is built by chaining together 2 or more .vox files (could be one as well).

Basically my logs will look like this:

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPromptswelcome.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPrompt11.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPrompt12.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 Lots of other stuff I don't care about

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 More stuff I don't care about

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 More of the same - I don't care about

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPrompt21.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/12345678 VoiceCallFlow c:MyPrompts\C1\IvrPrompt22.vox

AAA-PROD-IVR1 DL 01:46:34.405|FYI|69/1234567 Blablabla - today and tomorrow

How can I count just the Prompts (i.e. contiguous chain of *.vox files)?

How can I determine the average "size" of my prompts? - i.e. the average prompt size = 6 *.vox files

Thank you!!
A.C.

Re: Regex sought for consecutive-multi-line-search-joined-on-common-id display

kristian_kolb — Wed, 26 Mar 2014 09:02:38 GMT

I can see two things that might work for you;

a) use transaction like you already do, and subtract X from the automatically created field eventcount. This will work fine if you have a fixed number of events per transaction that you don't want to count, i.e. GCEV_DISCONNECT and USR_PLAYPROMPT.

... | eval prompt_size = eventcount - 2

b) if there is an unknown number of events in the transaction you do not want to count, you can do it like so;

host=*prod-ivr* *.vox | rex "FYI|(?<call_id_ivr>\S+)" |     stats c by call_id_ivr | stats avg(c) as avg_prompt