Correlate field from earlier event with error

[mfrost8]

I have a user who has a need to look for exceptions in a log file, and then figure out what order the error pertained to by looking for the most recently indexed previous event that defined a particular field.

That is, the exception itself won't say anything about the order that caused it, but then again, the line that has the order on it doesn't show the exception (or any error, really). This means I have to somehow tell Splunk to look backwards to find the data (field) that I really need to report on.

How can I do this rather odd request?

thanks

[mfrost8]

I'd thought about that, but there's not much to show. These are really horrible, user-generated "logs". Picture what is effectively a dump of 1 long row from a database (no values) followed by a line or two later (no datestamps!) a java exception.

[emiller42 · Accepted solution]

streamstats might actually help here. Add the following to your search (Change fields as needed)

| streamstats last(orderNumber) as last_order 

This will create a new field called last_order for each event which will have the last seen orderNumber up to the point of that event occurring.

So something like this:

_raw=orderNumber=123       last_order=123
_raw=some other message    last_order=123
_raw=orderNumber=456       last_order=456
_raw=another message       last_order=456
_raw=ERROR Message!        last_order=456

Which will associate the last seen order number with the error message even though there's no explicit correlation.

(Note, you can add a 'by' clause to the streamstats if you need to restrict the aggregation somehow, say by host)

Streamstats Documentation

[mfrost8]

Wow. Very nice. I looked at the streamstats command based on a previous post, but I found the fact that it said it was like stats to be confusing as I didn't want any kind of cumulative stats done on the events.

That was what I was looking for. Thanks.

[kristian_kolb]

You'd be able to get better help if you include some sample events. It's a bit unclear (to me) how the events are linked together...