I am trying to speed up my data model search for an alert that checks every 5 minutes (for the last 5 minutes) for "excessive" blocked inbound network connections (external IP's to the Internal RFC1918 IP space" I have the searching working just with the Data Model, but it still seems slow. Is there any way I can speed this up?
| `datamodel("Network_Traffic", "All_Traffic")`
| search All_Traffic.action="blocked" (All_Traffic.dest=10.0.0.0/8 OR All_Traffic.dest=172.16.0.0/12 OR All_Traffic.dest=192.168.0.0/16) AND NOT (All_Traffic.src="10.0.0.0/8" OR All_Traffic.src="172.16.0.0/12" OR All_Traffic.src="192.168.0.0/16")
| stats count dc(All_Traffic.dest) as dest_count, values(All_Traffic.dest) as "Destination IP" by All_Traffic.action, All_Traffic.src | rename All_Traffic.action as "Action", All_Traffic.src as "Source IP"
| search count>150
Don't use |datamodel or the macro.... use | tstats instead that is way faster! only downside for tstats is that you can't use a cidr in your where.
your query whould become something like:
| tstats summariesonly=t count dc(All_Traffic.dest) as dest_count, values(All_Traffic.dest) as dest from datamodel=Network_Traffic where All_Traffic.action="blocked" by All_Traffic.action, All_Traffic.src
| `drop_dm_object_name("All_Traffic")`
| search (dest=10.0.0.0/8 OR dest=172.16.0.0/12 OR dest=192.168.0.0/16) AND (src!="10.0.0.0/8" src!="172.16.0.0/12"
src!="192.168.0.0/16")
| where count>150d
| rename dest AS "Destination IP", action as "Action", src as "Source IP"
more info on tstats: http://docs.splunk.com/Documentation/Splunk/latest/SearchReference/Tstats