Log Filtering Performance Metrics or Performance Issues

[jasonchangbompa]

Hi, I have an open question about splunk "log filtering" performance.

Is there any document or technical report mentioning about performance metrics which clearly shows case-by-case (e.g. depending on number of cores, etc) filtering performance?

Nothing comes out of my mind at this point regarding this performance issue, yet. But I guess you know what I'm talking about...

Best Wishes, Sung Jin

[kristian_kolb]

Sorry, but I don't know what you are talking about.

First of all "log filtering" can mean a few things - many people would probably interpret a process of selectively discarding events prior to indexing. Others would think you're meaning searching for already indexed events.

As for discarding events prior to indexing the performance depends on the complexity of the regex, so there are no hard performance metrics to compare against. Depending on the setup of your Splunk infrastructure, this process can take place on either a heavy forwarder or an indexer, so even if you have very complex filtering rules, it may not affect the indexing/search performance at all.

When it comes to searching.. well.. you can write searches of varying degrees of efficiency, and still come up with the same results. It's all about knowledge of your data, and your skill in creating searches.

Hope this helps,

Kristian