Splunk Search

Why alert triggers false positive, but testing from a new Search produces correct results?

RNB
Path Finder

I am having an issue where I have created a search string that returns the correct results, but when used as an alert, it returns different results. I am attempting to identify all Cisco messages that have a severity level of -1-.

( -1- NOT (search OR "CALLMANAGER-6"))

This is the expected result from Search and from the Alert.

Jan 19 14:19:49 192.203.1.20 Wireless: *apfReceiveTask: Jan 19 19:19:46.819: #APF-1-ADD_TO_BLACKLIST_FAILED: apf_ms.c:5636 Unable to create exclusion-list entry for mobile a4:ee:57:22:e9:a3 

However, the Alert also triggers on the following types of events where -1- appears anywhere in the log entry (near end of line 3), despite excluding CALLMANAGER-6 which is the typical false positive.

Jan 19 14:06:27 192.203.1.21 456037: Jan 19 19:06:27.664 UTC : %CCM_CALLMANAGER-CALLMANAGER-6-DeviceRegistered: Device registered. Device name.:SEP44ADD9BC819C Device IP address [Optional].:10.42.31.22 Protocol.:SCCP Device type. [Optional]:495 Performance monitor object type:2 Device description [Optional].:OR-Res2-101 Load ID. [Optional]:SCCP69xx.9-4-1-3SR1 Associated directory numbers. [Optional].:3672 Device MAC address [Optional].:44ADD9BC819C IPAddressAttributes [Optional].:3 ActiveLoadId [Optional].:SCCP69xx.9-3-3-2 App ID:Cisco CallManager Cluster ID:StandAloneCluster Node ID:BAUCCMS01

I also tried a regex with no success, where the regex matches any string starting with % or #, includes -1-, ends with a colon.

* | regex_raw="[%,#]*.-1-*.:"

Any ideas how to get the Alert working the way I need it to?

Thank you
Randy

Tags (2)
0 Karma

MuS
Legend

Hi RNB,

I cannot solve this problem, but i can help you to find a different way to achieve this. If I get it correct you want to match the -1- in #APF-1-ADD_TO_BLACKLIST_FAILED? If this is correct, you can use field extraction like you tried and simply search for the values you're interested in. Try something like this:

your base search to get the events here | rex "\:\s[\#\%](?<myField>.*?)\:" | search myField="*-1-*" 

This will create a new field called myField (name can be changed to what ever) containing APF-1-ADD_TO_BLACKLIST_FAILED and CCM_CALLMANAGER-CALLMANAGER-6-DeviceRegistered based on the provided examples. The last | search will only return all myField values which contains a -1- .

After you got all working, setup automatic field extraction http://docs.splunk.com/Documentation/Splunk/6.2.1/Knowledge/Createandmaintainsearch-timefieldextract... and you no longer need to use rex.

hope this helps to get you started ...

cheers, MuS

Get Updates on the Splunk Community!

Thanks for the Memories! Splunk University, .conf24, and Community Connections

Thank you to everyone in the Splunk Community who joined us for .conf24 – starting with Splunk University and ...

.conf24 | Day 0

Hello Splunk Community! My name is Chris, and I'm based in Canberra, Australia's capital, and I travelled for ...

Enhance Security Visibility with Splunk Enterprise Security 7.1 through Threat ...

 (view in My Videos)Struggling with alert fatigue, lack of context, and prioritization around security ...