topic Re: Capture text between events in Dashboards & Visualizations

Capture text between events

Kindred — Tue, 10 Jun 2014 08:10:04 GMT

I have some logs being indexed that contain the output from another program, and Splunk indexes like this:

Event 1:

commandOuput=
/some/random/command blah blah blah

Event 2:

line1 output from the command

Event 3:

line2 output from the command

Event XX:

lineXX output from the command

Event Z:

commandExitcode=0

I want to capture and concatenate all text between the first event, and the last event (Z). I've tried using transaction and matched the start with startswith and end with endswith, but that only seems to keep the start and end event, not the events inbetween.

Is there an easier way of just extracting the raw data between two events?

Re: Capture text between events

MuS — Tue, 10 Jun 2014 08:20:56 GMT

Hi Kindred,

try something like this:

 your base search | rex field=_raw "Event\s1\:(?<myNewField).*)Event\sZ\:" | table myNewField

this will get everything between Event 1: and Event Z:.

hope this helps ...
cheers, MuS

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 08:25:34 GMT

Sorry didn't meant "Event .." was in the text, I was just identifying them as different events from Splunk's point of view.

Re: Capture text between events

MuS — Tue, 10 Jun 2014 08:35:37 GMT

well, then try:

 your base search | rex field=_raw "commandOuput=(?<myNewField).*)commandExitcode=0" | table myNewField

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 09:06:23 GMT

That doesn't seem to match anything. If I remove commandExitcode then it matches the first line in the first event, but nothing after.

Re: Capture text between events

MuS — Tue, 10 Jun 2014 09:36:10 GMT

My bad, there is a typo and a copy / paste error in it 😉

This one is tested with your provided example and it works:

"commandOuput=(\r)+(?<myField>(.+\r)+)(\r)+commandExitcode=0"

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 12:23:37 GMT

I can see what you're trying to do and I've done regex extractions before, but it still won't extract over multiple events - it only pulls out the first line of the first event.

Re: Capture text between events

MuS — Tue, 10 Jun 2014 12:38:26 GMT

Sorry it is way to hot in the office to have a clear thought 😉 Now I see your problem, this pasted example is not the raw test of one event, those are different events facepalm
So keep the transaction with startswith and endswith and create or use a common field for all the other events you need. Add this field to the transaction command like this:

transaction startswith="commandOuput=" endswith="commandExitcode=0" thenewfield

where the newfield cound be for example a regex for line1 output

hope this makes sense now

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 13:14:21 GMT

We don't have control over the "command output" that gets logged, so no way to tag it for consideration in the transaction - that's why I originally had trouble, I could match the start and end transaction events, but the text "in between" the events can't be correlated.

What I need is like saying "give me all the events between startswith=foo and endswith=bar regardless of whats in the events".

Re: Capture text between events

MuS — Tue, 10 Jun 2014 13:21:47 GMT

try using the _indextime field as additional transaction field if there is no other field in common over all events

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 13:29:31 GMT

_indextime changes over the course of time as the command output is logged, so won't help unfortunately.

Re: Capture text between events

MuS — Tue, 10 Jun 2014 13:46:21 GMT

take this run everywhere search and you will see that using _indextime works fine:

index=_internal source=*access.log | transaction clientip _indextime maxspan=5min

by using it with startswith and endswith you should be able to get what you want

Re: Capture text between events

Kindred — Tue, 10 Jun 2014 14:15:36 GMT

I tried, it doesn't change anything - it still only gives me the first event (startswith) and last events (endswith) - the events in between are not included.