Finding unique entries

jlixfeld — Thu, 17 Oct 2013 16:58:03 GMT

I have syslog data that looks like so:

2013-10-17T12:37:01.608054-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.3 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:02.367813-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.131 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:03.117860-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.3 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:03.867785-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.131 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:04.617843-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.3 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:05.367849-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny SCTP reverse path check from 10.10.23.2 to 10.10.22.131 on interface SYSTEMS-MANAGEMENT 2013-10-17T12:37:08.349020-04:00 fw01.77MowatAv01.YYZ %ASA-1-106021: Deny TCP reverse path check from 192.0.2.1 to 184.28.42.175 on interface MANAGEMENT

I want to run a one-off query against this syslog data to pull out the unique events based on the values of two fields within each event.

I'm having a hard time getting the IP addresses which I want to use as the unique values to match against other events. Assuming the field separator is a space character, the IP address fields would be fields 10 and 12 of the sample syslog output above.

In shell scripting land, I can simply use awk -F" " '{print $10" "$12)' but is there a way that is just as easy where I can specify the field delimiter and the field positions in Splunk?

Am I making any sense here?

Essentially what I want to see from this query is that based on the syslog sample above, the following is unique:

10.10.23.2 10.10.22.3
10.10.23.2 10 .10.22.131
192.0.2.1 184.28.42.175

The rex and regex stuff is far too complex for my little brain to comprehend, so I'm trying to see if there is a way I can use familiar tools, or concepts from familiar tools to achieve the same results.

Thanks in advance!

BTW: Using Splunk Enterprise 6.

Re: Finding unique entries

richgalloway — Thu, 17 Oct 2013 17:57:27 GMT

Rex is pretty simple once you get the hang of it. Try something like this:

... | rex field=_raw "(?<arg1>.*?)\s(?<arg2>.*?)\s(?<arg3>.*?) Deny TCP reverse path check from (?<From>.*?) to (?<To>.*?) on interface (?<interface>.*?)" | dedup From, To | table From, To

Someone else may have a more elegant solution.

topic Finding unique entries in Splunk Search

Finding unique entries

Re: Finding unique entries