topic Re: sub-search and destIP foreach srcIP in Splunk Search

sub-search and destIP foreach srcIP

Gilgalidd — Mon, 12 Aug 2013 12:11:30 GMT

Hello,

I would like to obtain a complete list of all connection.

for exemple :

SRC         | DST         |PORT
a.a.a.a     | z.z.z.z     | tcp 22
            | x.x.x.x     | tcp 8080
b.b.b.b     | x.x.x.x     | tcp 80
...

For that, I've made two search, one for list all src IP and the second for the dst IP :

source="toto.log"
  | rex max_match=100 "\binside:(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
  | stats values(ip) as ip_list


source="toto.log" inside:X.X.X.X
  | rex max_match=100 "\boutside:(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
  | stats values(ip) as ip_list

But I don't know how do a sub-search to have a result like I've explain.

My log look like:

Aug  1 00:00:09 x.x.x.x %FWSM-4-106023: Deny udp src inside:x.x.x.x/50464 dst outside:x.x.x.x/53 by access-group "inside_access_in" [0x0, 0x0]

Can I do what I want ? If yes, how ? And Is it the best way to obtain the result ?

Thanks for reading.

Re: sub-search and destIP foreach srcIP

antlefebvre — Mon, 12 Aug 2013 13:08:05 GMT

Create field extractions instead of doing a rex in search. Then you can do a search line this:

source="toto.log" | stats list(dstip),list(dstport) by srcip

Re: sub-search and destIP foreach srcIP

kristian_kolb — Mon, 12 Aug 2013 14:39:21 GMT

Assuming that you have extracted the protocol, src_ip, dst_ip and dst_port as fields (either through conf files or with rex) you can do this by concatenating the destination fields together;

...| eval destination = dst_ip . " " . protocol . " " . dst_port 
| stats values(destination) by src_ip

The function values give the distinct values for a field. If using list you get all of them, which may include duplicates.

Re: sub-search and destIP foreach srcIP

kristian_kolb — Mon, 12 Aug 2013 14:40:41 GMT

With this approach, there is no connection between the list of IP's and the list of ports. They will be sorted independently.

Re: sub-search and destIP foreach srcIP

antlefebvre — Mon, 12 Aug 2013 15:44:07 GMT

Thank you much. I was unaware that list sorted independently.

Re: sub-search and destIP foreach srcIP

kristian_kolb — Mon, 12 Aug 2013 16:15:46 GMT

oops, I might have been a bit too quick there. According to docs lists will be sorted by the order in which they are returned. However making such a list is just like a table. Sorry for my confusing things.

Re: sub-search and destIP foreach srcIP

Gilgalidd — Tue, 13 Aug 2013 08:12:58 GMT

Thanks for your reply, but i don't understand how use this with my log. Can you give me more information to get protocol, port and other fields ?

Re: sub-search and destIP foreach srcIP

kristian_kolb — Tue, 13 Aug 2013 08:26:11 GMT

You'll need to extract the relevant portions of the event into so-called 'fields'. you can do that with rex as part of a search query (which you have already done), or put (more or less) the same logic into config files, so that the fields are automatically extracted.

Start at this page, and follow some of the links to understand how that is performed;

http://docs.splunk.com/Documentation/Splunk/latest/Tutorial/Usefieldstosearch

Re: sub-search and destIP foreach srcIP

Gilgalidd — Tue, 13 Aug 2013 14:32:04 GMT

Thanks a lot for your help !

source="toto.log"
  | rex max_match=100 "\bsrc (?<Sint>\w{1,99}):(?<Sip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/(?<Sport>\d{1,5})\b"
  | rex max_match=100 "\bdst (?<Dint>\w{1,99}):(?<Dip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/(?<Dport>\d{1,5})\b"
  | eval src=Sint .":".Sip."/".Sport 
  | eval dst=Dint .":".Dip."/".Dport
  | stats values(src) by dst

give me a nice result