There are many ways to look at search performance, particularly of windows event log data. You should get comfortable with understanding the search job properties page. In particular, look at the pha...
See more...
There are many ways to look at search performance, particularly of windows event log data. You should get comfortable with understanding the search job properties page. In particular, look at the phase1 search and the scanCount. scanCount is the number of events that were scanned to return the results. With winevent log data particulary, you should understand that in order for the search to know if the process_name field is not what you want, it has to look at all events because process_name is a field that is is mapped by the Windows TA. Minimising the number of events you look at (scanCount) will always help performance. Look at this presentation that shows how to use TERM() effectively. That can be a significant benefit to your searches. https://conf.splunk.com/files/2020/slides/PLA1089C.pdf For example, your initial search index=windows source=XmlWinEventLog:Security process_name=ipconfig.exe can most likely be significantly improved just by writing index=windows source=XmlWinEventLog:Security TERM(ipconfig) process_name=ipconfig.exe because instead of pulling every event out to see if the windows TA has mapped a piece of the raw event to the process_name field, it will ONLY look at the events that have the term ipconfig in the raw event, so given that ipconfig will be a less frequently used command, your scanCount will drop significantly. In the search log from the inspect job page, search for LISPY and you can see how the parser has interpreted your search. In your other example of the != vs NOT, take a look at the phase0 search in the job properties. You will no doubt see a significant different in the expanded search. There are other forms of "filters", such as subsearches and lookups, but I would say that there is not often a one-size-fits-all approach to optimising your searches. It frequently depends on your data and the event count and cardinality of values you get back for fields you're trying to exclude. Lookups are often a good way to filter data, particularly when your data is still being searched in the index tier, i.e. before a transforming command has sent the data to the search head. So, it can be more efficient to do this type of logic index=windows source=XmlWinEventLog:Security
| lookup process_names.csv process_name OUTPUT is_this_one_i_want
| where isnotnull(is_this_one_i_want) which will then drop all events where process_name is included in your lookup. Note that this is a poor example, as it would grab all events and then filter, but the point is that it can be more efficient to first limit your data set in the primary search then filter using a lookup to remove other events rather than writing an up-front really complex set of conditions.