It seems like if you I have a numeric multivalued field, I should be able to use eval to take the max and min of the values per row.
For example, I have a 'bytes' field on my events. i form those events into transactions and now i have a nice multivalued 'bytes' field on my transaction rows.
From here I'd like to get the max and min values of bytes per row, ie so I end up with a single 'maxBytes' number per transaction row.
<my search> | transaction user | eval maxBytes=max(bytes)
However when I do this I end up with a multiValued maxBytes, and its exactly as though I had just done:
<my search> | transaction user | eval maxBytes=bytes
Is there a reason for why it works this way or is there a workaround for it?
After re-reading the docs, it does seem like this would be a nice feature request. I think you have to use stats
and not eval
when you are using the max()
function this way. It seems like you have to have a list of values for max
in eval
where as stats
max()
works fine with multi-valued fields. (There does seem to be some lacking features for multi-value fields like this. Another one would be that you can't simply add a value to a multi-value field without some ugly delimiter-based hacks.)
Perhaps this would be the best fit for your situation:
... | streamstats max(bytes) as maxBytes window=1
BTW. Make sure that use include mvlist="bytes"
on your transaction
command, or you will get a unique list of values for bytes
instead of a one-for-one listing of your bytes
values. (I guess this really only matters if your doing sum(bytes)
or something like that, but this seems easy to overlook so I figured I point it out.)
First, transaction does not take the max, but instead lists all the value for each field in the order it encounters them much like:
| stats list(field1) AS field1
Second, max is NOT used on a multivalue field, but rather on a comma-separated list like:
| eval n=max(1, 3, 6, 7, "foo", foo2)
where "foo" is evaluated as greater than 0.
Third, if you want the max of a multivalued field, use stats like:
| stats max(field1) AS field1
Fourth, the accepted answer to this question about re-reading the docs and being a "nice feature request" has SO many inaccuracies that it really should be deleted.
After re-reading the docs, it does seem like this would be a nice feature request. I think you have to use stats
and not eval
when you are using the max()
function this way. It seems like you have to have a list of values for max
in eval
where as stats
max()
works fine with multi-valued fields. (There does seem to be some lacking features for multi-value fields like this. Another one would be that you can't simply add a value to a multi-value field without some ugly delimiter-based hacks.)
Perhaps this would be the best fit for your situation:
... | streamstats max(bytes) as maxBytes window=1
BTW. Make sure that use include mvlist="bytes"
on your transaction
command, or you will get a unique list of values for bytes
instead of a one-for-one listing of your bytes
values. (I guess this really only matters if your doing sum(bytes)
or something like that, but this seems easy to overlook so I figured I point it out.)
I downvoted this post because there are several inaccuracies with this answer. A list of values in a field IS a multi-value field. also, you can add a value to a multi-value field using mvappend
in eval
. streamstats is not best in his situation. transaction does not give a unique list, but rather lists all values like list
in stats
.
great answer, Lowell. Fit my needs aswell 🙂
Thanks lowell. Extremely helpful. And I had not come across mvlist="bytes" yet, or noticed that transaction was only keeping distinct values by default.