Why are alerts constantly getting triggered by false positives?

[metylkinandrey]

Good afternoon! I have a problem setting up alerts. Most allerts, with the exception of one, are processed incorrectly. Alerts are processed by scheduled, last 1 minute. By wrong - I mean their false positives, that is, they are constantly triggered by scheduled, even if the request conditions are not met during this period of time.

Despite the fact that requests in alerts work out correctly - as we need, I am convinced that the problem is in the syntax, since the settings for the correct alert and the problematic alerts are the same.

Examples:

Alert that works fine:

index="main" sourcetype="testsystem-script11"

| transaction maxpause=10m srcMsgId Correlation_srcMsgId messageId

| table _time srcMsgId Correlation_srcMsgId messageId duration eventcount

| fields _time srcMsgId Correlation_srcMsgId messageId duration eventcount

| sort srcMsgId _time

| streamstats current=f window=1 values(_time) as prevTime by subject

| eval timeDiff=_time-prevTime

| delta _time as timeDiff

| where (timeDiff)>1

An example of a problematic alert (I thought that the problem was in Cyrillic characters, but I tried without them, it does not help):

index="main" sourcetype="testsystem-script99" resultcode>0

| eval srcMsgId_Исх_Сообщения=if(len('Correlation_srcMsgId')==0 OR isnull('Correlation_srcMsgId'),'srcMsgId','Correlation_srcMsgId')

| eval timeValue='eventTime'

| eval time=strptime(timeValue,"%Y-%m-%dT%H:%M:%S.%3N%Z") | sort -eventTime | streamstats values(time) current=f window=1 as STERAM_RESULT global=false by srcMsgId_Исх_Сообщения

| eval diff=STERAM_RESULT-time

| stats list(diff) as TIME_DIF list(eventTime) as eventTime list(srcMsgId) as srcMsgId_Бизнес_Сообщения list(routepointID) as routepointID count as Кол_Сообщений by srcMsgId_Исх_Сообщения

[richgalloway]

To properly help you, we need to know the alert trigger settings - not just the search query.  Sharing the complete savedsearches.conf stanzas would be very helpful.

[metylkinandrey]

I will try to attach screenshots

[richgalloway · Accepted solution]

The first query has a clear alert condition: any result with a timeDiff value greater than 1.  There is no clear alert condition in the second query.  The stats command will almost always return *something* as long as the fields in the by clause are not null.  Try re-writing the second query to produce a more explicit result.

[metylkinandrey]

Thank you! Edited the request according to your advice, it works.