How to capture multiline events (with a regex)?

[skender27]

Hi,

I am trying to capture the multiline events from a Weblogic-similar log which satisfies all three conditions below:
- start with "Entering Strategy" string
- contains at least one [ACTIVE]
- finish with "Exiting strategy" string

I experimented with the regex101 (Python syntax), but I always get unclear results...

[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)': Entering strategy : PRESCORE : Jun 19, 2015 9:58:12 AM
[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)': IN : Setup (Process Flow Object)
[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)':  IN : Setup (Perform Process)
[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)':    IN : Activation Modules (Segmented Process. Exiting strategy)
[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)'
[ACTIVE] ExecuteThread: '31' for queue: 'weblogic.kernel.Default (self-tuning)':...
...

Any suggestions?

Thanks a lot,
Skender Kollcaku

[woodcock]

Use this in your props.conf:

[yourSourceTypeHere]
SHOULD_LINEMERGE=false
LINE_BREAKER = ([\r\n]+)\[[^\r\n]*?Entering\s+strategy
MUST_BREAK_AFTER = Exiting\s+strategy\)(?=[\r\n])

Then you need to deploy it to your indexers (or Heavy Forwarders) and restart each Splunk instance before it will take effect. You might also look into sending the "other" events into nullQueue; what you have done so far simply detaches them from the events you would like to keep but they still exist as their own separate events:

http://docs.splunk.com/Documentation/Splunk/6.2.4/Forwarding/Routeandfilterdatad

BTW, this is also where you would trigger your 3rd requirement ("contains at least one [ACTIVE]") which the props.conf does not include. You just send those without "[ACTIVE]" to nullQueue, too.

[maciep]

not sure if you're just trying to do a field extraction or actually index the data? But maybe something like this is helpful?

(?<thread>Entering strategy[\s\S]+\[ACTIVE\][\s\S]+Exiting strategy)