topic Improve search performance in Splunk Enterprise Security

Improve search performance

shaquibk — Mon, 27 Dec 2021 10:13:20 GMT

Hi All,

I need to improve the performance of my below search, which currently completes in about 132sec. The search looks for last 7 days data from firewall logs.

Search:

index="xxx" src_ip !="a.b.c.d/26" src_ip !="x.y.z.w/26" src_zone!=ABCD src_zone!=ABCDE (dest_zone = "ABCD" OR (dvc_name IN ("qwerty","abcd","xyz","asdf") AND dest_zone="XYZ")) app IN (ldap,rmi-iiop) | lookup some_lookup ip as src_ip OUTPUT matched | search matched!="yes" | stats count by src_ip,action,date_mday | stats count by src_ip,action | search (action=allowed OR (action=blocked AND count>1))

Thanks in advance.

Regards,

Shaquib

Re: Improve search performance

scelikok — Mon, 27 Dec 2021 10:52:00 GMT

Hi @shaquibk,

You can try below, I changed lookup method;

index="xxx" src_ip !="a.b.c.d/26" src_ip !="x.y.z.w/26" src_zone!=ABCD src_zone!=ABCDE (dest_zone = "ABCD" OR (dvc_name IN ("qwerty","abcd","xyz","asdf") AND dest_zone="XYZ")) app IN (ldap,rmi-iiop) [| inputlookup some_lookup | fields ip | rename ip as src_ip | format ] | stats count by src_ip,action,date_mday | stats count by src_ip,action | search (action=allowed OR (action=blocked AND count>1))

Re: Improve search performance

shaquibk — Tue, 28 Dec 2021 09:44:57 GMT

Hi @scelikok

The below method somehow doesn't return any results at all.

Thanks,

Shaquib

Re: Improve search performance

isoutamo — Tue, 28 Dec 2021 10:48:06 GMT

maybe you should use this https://docs.splunk.com/Documentation/CIM/5.0.0/User/Overview with data model acceleration? There is https://docs.splunk.com/Documentation/CIM/5.0.0/User/NetworkTraffic for this kind of use.

r. Ismo