What is the quickest way to find 100 max values of "Q" on huge log file?

[indeed_2000]

Hi

What is the quickest way to find 100 max values of "Q" on huge log file?

here is my query:

index="myindex" |  rex "Q\[(?<Q>\d+) | stats max(Q)

here is the log:

13:58:34.999  Q[16]

Any idea?

Thanks

[ITWhisperer]

index="myindex" | rex "Q\[(?<Q>\d+)" | stats max(Q)

[indeed_2000]

@ITWhispererThanks for answer. what is the different?

[ITWhisperer]

The difference is mine will work, yours won't! 😀

"status" isn't a valid SPL command

Adding the field name to the rex expression identifies what the field is to be called.

[indeed_2000]

@ITWhisperer actually i have performance issue, original post modified, I had typo.

[ITWhisperer]

Rather than extracting the Q field at search time, you could consider extracting it at index time (transform/props .conf). That would at least remove the rex processing when trying to determine the max.

[ITWhisperer]

You could also try creating a summary index with the max values every minute for example, then look for the max in the summary index events rather than the raw events.

[indeed_2000]

@ITWhisperer actually this is summary index 🙂 and still huge!
I try to use transform too but increase index time drastically !

Any other idea?

[ITWhisperer]

Try a summary of the summary in larger timeslices.