sub-search and destIP foreach srcIP

[Gilgalidd]

Hello,

I would like to obtain a complete list of all connection.

for exemple :

SRC         | DST         |PORT
a.a.a.a     | z.z.z.z     | tcp 22
            | x.x.x.x     | tcp 8080
b.b.b.b     | x.x.x.x     | tcp 80
...

For that, I've made two search, one for list all src IP and the second for the dst IP :

source="toto.log"
  | rex max_match=100 "\binside:(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
  | stats values(ip) as ip_list


source="toto.log" inside:X.X.X.X
  | rex max_match=100 "\boutside:(?<ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\b"
  | stats values(ip) as ip_list

But I don't know how do a sub-search to have a result like I've explain.

My log look like:

Aug  1 00:00:09 x.x.x.x %FWSM-4-106023: Deny udp src inside:x.x.x.x/50464 dst outside:x.x.x.x/53 by access-group "inside_access_in" [0x0, 0x0]

Can I do what I want ? If yes, how ? And Is it the best way to obtain the result ?

Thanks for reading.

[kristian_kolb · Accepted solution]

Assuming that you have extracted the protocol, src_ip, dst_ip and dst_port as fields (either through conf files or with rex) you can do this by concatenating the destination fields together;

...| eval destination = dst_ip . " " . protocol . " " . dst_port 
| stats values(destination) by src_ip

The function values give the distinct values for a field. If using list you get all of them, which may include duplicates.

/K

[Gilgalidd]

Thanks a lot for your help !

source="toto.log"
  | rex max_match=100 "\bsrc (?<Sint>\w{1,99}):(?<Sip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/(?<Sport>\d{1,5})\b"
  | rex max_match=100 "\bdst (?<Dint>\w{1,99}):(?<Dip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3})\/(?<Dport>\d{1,5})\b"
  | eval src=Sint .":".Sip."/".Sport 
  | eval dst=Dint .":".Dip."/".Dport
  | stats values(src) by dst

give me a nice result

[kristian_kolb]

You'll need to extract the relevant portions of the event into so-called 'fields'. you can do that with rex as part of a search query (which you have already done), or put (more or less) the same logic into config files, so that the fields are automatically extracted.

Start at this page, and follow some of the links to understand how that is performed;

http://docs.splunk.com/Documentation/Splunk/latest/Tutorial/Usefieldstosearch

[Gilgalidd]

Thanks for your reply, but i don't understand how use this with my log. Can you give me more information to get protocol, port and other fields ?

[antlefebvre]

Create field extractions instead of doing a rex in search. Then you can do a search line this:

source="toto.log" | stats list(dstip),list(dstport) by srcip

[kristian_kolb]

oops, I might have been a bit too quick there. According to docs lists will be sorted by the order in which they are returned. However making such a list is just like a table. Sorry for my confusing things.

[antlefebvre]

Thank you much. I was unaware that list sorted independently.

[kristian_kolb]

With this approach, there is no connection between the list of IP's and the list of ports. They will be sorted independently.