Splunk Search

Why alert triggers false positive, but testing from a new Search produces correct results?

Path Finder

I am having an issue where I have created a search string that returns the correct results, but when used as an alert, it returns different results. I am attempting to identify all Cisco messages that have a severity level of -1-.

( -1- NOT (search OR "CALLMANAGER-6"))

This is the expected result from Search and from the Alert.

Jan 19 14:19:49 Wireless: *apfReceiveTask: Jan 19 19:19:46.819: #APF-1-ADD_TO_BLACKLIST_FAILED: apf_ms.c:5636 Unable to create exclusion-list entry for mobile a4:ee:57:22:e9:a3 

However, the Alert also triggers on the following types of events where -1- appears anywhere in the log entry (near end of line 3), despite excluding CALLMANAGER-6 which is the typical false positive.

Jan 19 14:06:27 456037: Jan 19 19:06:27.664 UTC : %CCM_CALLMANAGER-CALLMANAGER-6-DeviceRegistered: Device registered. Device name.:SEP44ADD9BC819C Device IP address [Optional].: Protocol.:SCCP Device type. [Optional]:495 Performance monitor object type:2 Device description [Optional].:OR-Res2-101 Load ID. [Optional]:SCCP69xx.9-4-1-3SR1 Associated directory numbers. [Optional].:3672 Device MAC address [Optional].:44ADD9BC819C IPAddressAttributes [Optional].:3 ActiveLoadId [Optional].:SCCP69xx.9-3-3-2 App ID:Cisco CallManager Cluster ID:StandAloneCluster Node ID:BAUCCMS01

I also tried a regex with no success, where the regex matches any string starting with % or #, includes -1-, ends with a colon.

* | regex_raw="[%,#]*.-1-*.:"

Any ideas how to get the Alert working the way I need it to?

Thank you

Tags (2)
0 Karma



I cannot solve this problem, but i can help you to find a different way to achieve this. If I get it correct you want to match the -1- in #APF-1-ADD_TO_BLACKLIST_FAILED? If this is correct, you can use field extraction like you tried and simply search for the values you're interested in. Try something like this:

your base search to get the events here | rex "\:\s[\#\%](?<myField>.*?)\:" | search myField="*-1-*" 

This will create a new field called myField (name can be changed to what ever) containing APF-1-ADD_TO_BLACKLIST_FAILED and CCM_CALLMANAGER-CALLMANAGER-6-DeviceRegistered based on the provided examples. The last | search will only return all myField values which contains a -1- .

After you got all working, setup automatic field extraction http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Createandmaintainsearch-timefieldextract... and you no longer need to use rex.

hope this helps to get you started ...

cheers, MuS

Get Updates on the Splunk Community!

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more with ITSI’s ...

Accelerate Service Onboarding, Decomposition, Troubleshooting - and more! Faster Time to ValueManaging and ...

New Release | Splunk Enterprise 9.3

Admins and Analyst can benefit from:  Seamlessly route data to your local file system to save on storage ...

2024 Splunk Career Impact Survey | Earn a $20 gift card for participating!

Hear ye, hear ye! The time has come again for Splunk's annual Career Impact Survey!  We need your help by ...