This will be very interesting or boring, it can only be one!
I have an extracted field: CFErrorCodeMessagesCode
This can contain one of many possible values, e.g. "Code (216)" "Code (9999)" e.g. "Code (xxxx)"
Normally, I have a spreadsheet that creates me a large query to run, for an alert on a cron schedule, that holds a row for each code. e.g with 3 codes the string it generates looks like this:
earliest=-10d index="cbeprodvteclogs" CFErrorCodeMessagesCode="Code (1)" | timechart count(CFErrorCodeMessagesCode) as cnt | trendline sma5(cnt) as cnt2 | eval spike=if(if(cnt <= 10, 0, cnt) > cnt2 * 3 , 1, 0) | eval CFECM="Code (1)" | FIELDS CFECM, cnt, cnt2, spike | append [ search
earliest=-10d index="cbeprodvteclogs" CFErrorCodeMessagesCode="Code (106)" | timechart count(CFErrorCodeMessagesCode) as cnt | trendline sma5(cnt) as cnt2 | eval spike=if(if(cnt <= 10, 0, cnt) > cnt2 * 3 , 1, 0) | eval CFECM="Code (106)" | FIELDS CFECM, cnt, cnt2, spike ] | append [ search
earliest=-10d index="cbeprodvteclogs" CFErrorCodeMessagesCode="Code (9999)" | timechart count(CFErrorCodeMessagesCode) as cnt | trendline sma5(cnt) as cnt2 | eval spike=if(if(cnt <= 10, 0, cnt) > cnt2 * 3 , 1, 0) | eval CFECM="Code (9999)" | FIELDS CFECM, cnt, cnt2, spike ] | where strftime(_time, "%Y-%m-%d") = strftime(now(), "%Y-%m-%d") and spike=1
The idea is to alert us of obscure spikes using sma5 over a 10 day data window, it's very effective and even better when sourced from a spreadsheet as I can tune every aspect of it via the columns.
Another alert I have is for codes the application pushes out that aren't in the SMA query (above).
I have 34 codes on one sheet/alert, this runs ok, on another sheet I have almost 60, and the performance has degraded since building up the list, and also since ingesting more data in to the index, but I don't see how this matters because I am explicitly specifying the code I need to pull.
I was considering putting the data to its own index, but this would complicate all sorts of stuff I've developed.
What I was wondering was: Is it possible to just have a single line that dynamically builds me a list? Something like the below query (which doesn't work, nor other variations I've tried)
earliest=-10d index="cbeprodswtlogs" CFErrorCodeMessagesCode=* | timechart count as cnt by CFErrorCodeMessagesCode | trendline sma5(cnt) as cnt2 | eval spike=if(if(cnt <= 50, 0, cnt) > cnt2 * 2 , 1, 0) | FIELDS CFErrorCodeMessagesCode, cnt, cnt2, spike
This would allow me to not have to maintain a spreadsheet and a separate alert for codes that aren't included already, it may also speed it up.
Thanks in advance.
... View more