Here's one I came up with. The idea is to cluster events over the last N hours, and then see if there are any clusters that only consist of events in the last 5 minutes. Those are your new type of events -- they don't look like anything seen in the last N hours.
Search back 10 hours; cluster results; for each cluster, keep the all _raw values, the oldest time, and the size; now, keep only those clusters that the oldest event in it was newer than 5 minutes; sort by size to get the smallest new clusters first.
earliest=-10h | cluster t=0.7 labelonly=t showcount=t
| stats values(_raw) as raw last(_time) as time last(cluster_count) as size by cluster_label
| eval minute5=now()-5*60
| where time > minute5
| sort size
| fields size, cluster_label, raw
You can run this as an alert to run every 5 minutes. You can tweak the initial hours and the 5 minute range as needed, and the t=0.7 value as well. If there are too many new clusters, increase the 10 hours window to prevent false positives of events that occur, for example, every 15 hours. If there are still too many new clusters, decrease the value of t (e.g., 0.6), so that clustering is more loose, and only more radically different events will be noted.
Hope that helps.
... View more
