Hi, splunk experts.
I'm using Splunk App for VMware 2.0 to collect data from my vSphere environment.
and I'm having difficulties from excluding certain file to be indexed.
Since vCenter vpxd-profile log file is big, I decided to exclude it from indexing.
At vCenter's Splunk_TA_vcenter folder, I copied props.conf and transforms.conf from default folder to local folder.
I confirmed that "TRANSFORMS-null" at vpxd-profile is not commented at props.conf.
But vpxd-*.log and vpxd-profile.log are still indexed.
I have inputs.conf, props.conf and transforms.conf files at C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_vcenter\local
Is there something should be done additionally?
excerpt from props.conf
[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd
LINE_BREAKER = ([\r\n]+\**)\[?\d{4}-\d{2}-\d{2}[T\s]\d{2}:\d{2}:\d{2}(?:\.\d{3})?(?:[\+\-]\d{2}\:\d{2})?Z?\s+\[?
MAX_TIMESTAMP_LOOKAHEAD = 80
SHOULD_LINEMERGE = true
#TRANSFORMS-null1 = vmware_vpxd_level_null
#TRANSFORMS-null4 = vmware_vpxd_retrieveContents_null
#TRANSFORMS-null5 = vmware_vpxd_null
[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-alert-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-alert
MAX_TIMESTAMP_LOOKAHEAD = 80
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+\**)\[?\d{4}-\d{2}-\d{2}[T\s]\d{2}:\d{2}:\d{2}(?:\.\d{3})?(?:[\+\-]\d{2}\:\d{2})?Z?\s+\[?
#TRANSFORMS-null2 = vmware_vpxd_level_null,vmware_vpxd_level_null2
#These files are to be parsed as single line events, always
[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-profiler-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-profiler
TIME_PREFIX = \[
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
EXTRACT-extract_kv_pairs = (?<key>.+)[\s](?<value>[^\s]+)$
TRANSFORMS-null3 = vmware_vpxd_level_null,vmware_vpxd_level_null2
excerpt from transforms.conf
#NullQueues
[vmware_vpxd_level_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\s(verbose|trivia)
[vmware_vpxd_retrieveContents_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\sinfo.*?task-internal.*?vmodl\.query\.PropertyCollector\.retrieveContents
[vmware_vpxd_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\s(verbose|trivia|info.*?task-internal.*?vmodl\.query\.PropertyCollector\.retrieveContents)
P.S.
I have successfully blocked vpxd-profil log with blacklisting it at inputs.conf.
But since inputs.conf is created automatically, controlling with nullQueue is wiser, I believe 🙂
Thank you.
follow-up to my own question.
I modified transforms.conf myself and now it works 🙂
I don't know what part of original conf prevented from exclusion.. but its ok
Splunk rocks!
excerpt of props.conf
[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-profiler-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-profiler
TIME_PREFIX = \[
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
EXTRACT-extract_kv_pairs = (?<key>.+)[\s](?<value>[^\s]+)$
TRANSFORMS-null3 = vpxd_profiler_death
excerpt of transforms.conf
#NullQueues
[vpxd_profiler_death]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = .
Thank you.