Splunk Search

Improve Speed of Correlation Search

cfloquet
Path Finder

Hello, thank you for taking the time to consider my question. 

I currently have a working SPL search that retrieves IPv4 addresses from a CSV using an inputlookup function, which works tremendously fast when operating by itself, however when I plug that inputlookup into a larger outer search that would correlate those values with destination IPv4s seen and reported by our firewall provider it takes much much longer for those results to actually appear (usually 2> minutes total runtime, and that's only using the suspicious IPs in the CSV from just the day before...)

Ideally this search would take less than a minute to complete, comparing around 25,000-30,000 IPv4s from the CSV with the several hundred that are reported by the firewall every 10 minutes or so. 

The syntax for the search is below:

 

 

index=firewall earliest=-10m@m latest=now vsys_name=Browser 
[| inputlookup phishCatch.csv | rename "IPv4" as dest_ip | table dest_ip]
| eval totalMBin=round(bytes_in/1024,2)
| rename generated_time as "Time Received" user as "Username" client_ip as "Source IPv4 Address" action as "Action Taken" totalMBin as "Total MB In" dest_ip as "Suspicious IPv4"
| table "Time Received", "Username","Source IPv4 Address","Suspicious IPv4","Total MB In","Action Taken"

 

 

 

I'm guessing that I will have to use some sort of acceleration to improve the speed, but I'm very much a Splunk novice and don't really understand datamodels or how Splunk acceleration actually works. 

Any advice on how best to proceed and improve the efficiency and speed of this search would be greatly appreciated! Thanks in advance

Labels (1)
0 Karma
Get Updates on the Splunk Community!

Technical Workshop Series: Splunk Data Management and SPL2 | Register here!

Hey, Splunk Community! Ready to take your data management skills to the next level? Join us for a 3-part ...

Spotting Financial Fraud in the Haystack: A Guide to Behavioral Analytics with Splunk

In today's digital financial ecosystem, security teams face an unprecedented challenge. The sheer volume of ...

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability

Solve Problems Faster with New, Smarter AI and Integrations in Splunk Observability As businesses scale ...