I have a user who has a need to look for exceptions in a log file, and then figure out what order the error pertained to by looking for the most recently indexed previous event that defined a particular field.
That is, the exception itself won't say anything about the order that caused it, but then again, the line that has the order on it doesn't show the exception (or any error, really). This means I have to somehow tell Splunk to look backwards to find the data (field) that I really need to report on.
How can I do this rather odd request?
You'd be able to get better help if you include some sample events. It's a bit unclear (to me) how the events are linked together...
streamstats might actually help here. Add the following to your search (Change fields as needed)
| streamstats last(orderNumber) as last_order
This will create a new field called last_order for each event which will have the last seen orderNumber up to the point of that event occurring.
So something like this:
_raw=orderNumber=123 last_order=123 _raw=some other message last_order=123 _raw=orderNumber=456 last_order=456 _raw=another message last_order=456 _raw=ERROR Message! last_order=456
Which will associate the last seen order number with the error message even though there's no explicit correlation.
(Note, you can add a 'by' clause to the streamstats if you need to restrict the aggregation somehow, say by host)
Wow. Very nice. I looked at the streamstats command based on a previous post, but I found the fact that it said it was like stats to be confusing as I didn't want any kind of cumulative stats done on the events.
That was what I was looking for. Thanks.
I'd thought about that, but there's not much to show. These are really horrible, user-generated "logs". Picture what is effectively a dump of 1 long row from a database (no values) followed by a line or two later (no datestamps!) a java exception.