We have a "complicated" directory structure for logging. I wanted to understand the impact on performance for Splunk Forwarder for the below case.
Lets say our logs have the below structure /foo/bar*/*/abc/def/ghi*.log and /foo/bar*/*/abc/def/jkl*.log
Now if we use the above wildcard structure for monitoring purpose, Splunk will translate it as
[monitor://foo/]
whitelist: <appropriately set for ghi*.log and jkl*.log files]
This will effectively monitor the directory and all the subdirectories inside /foo/. And as per my understanding, whenever a new files is added or updated anywhere inside /foo/ it will be compared against the whitelist by Splunk forwarder to decide whether to index it or not.
The number of files eligible for logging is not a lot (hardly 3-4). But the number of files that can be created inside /foo/ (all of which are eligible for whitelist comparison) can be in thousands.
Can anyone give a rough estimate of [if understood and known] the impact of this kind of directory structure will have on Splunk forwarder performance.
... View more