cannot redirect log file to nullQueue

[tomoyagoto]

Hi, splunk experts.


I'm using Splunk App for VMware 2.0 to collect data from my vSphere environment.

and I'm having difficulties from excluding certain file to be indexed.



Since vCenter vpxd-profile log file is big, I decided to exclude it from indexing.



At vCenter's Splunk_TA_vcenter folder, I copied props.conf and transforms.conf from default folder to local folder.

I confirmed that "TRANSFORMS-null" at vpxd-profile is not commented at props.conf.

But vpxd-*.log and vpxd-profile.log are still indexed.



I have inputs.conf, props.conf and transforms.conf files at C:\Program Files\SplunkUniversalForwarder\etc\apps\Splunk_TA_vcenter\local

Is there something should be done additionally?

---

excerpt from props.conf

[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd
LINE_BREAKER = ([\r\n]+\**)\[?\d{4}-\d{2}-\d{2}[T\s]\d{2}:\d{2}:\d{2}(?:\.\d{3})?(?:[\+\-]\d{2}\:\d{2})?Z?\s+\[?
MAX_TIMESTAMP_LOOKAHEAD = 80
SHOULD_LINEMERGE = true
#TRANSFORMS-null1 = vmware_vpxd_level_null
#TRANSFORMS-null4 = vmware_vpxd_retrieveContents_null
#TRANSFORMS-null5 = vmware_vpxd_null

[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-alert-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-alert
MAX_TIMESTAMP_LOOKAHEAD = 80
SHOULD_LINEMERGE = false
LINE_BREAKER = ([\r\n]+\**)\[?\d{4}-\d{2}-\d{2}[T\s]\d{2}:\d{2}:\d{2}(?:\.\d{3})?(?:[\+\-]\d{2}\:\d{2})?Z?\s+\[?
#TRANSFORMS-null2 = vmware_vpxd_level_null,vmware_vpxd_level_null2

#These files are to be parsed as single line events, always
[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-profiler-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-profiler
TIME_PREFIX = \[
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
EXTRACT-extract_kv_pairs = (?<key>.+)[\s](?<value>[^\s]+)$
TRANSFORMS-null3 = vmware_vpxd_level_null,vmware_vpxd_level_null2

---

excerpt from transforms.conf

#NullQueues
[vmware_vpxd_level_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\s(verbose|trivia)

[vmware_vpxd_retrieveContents_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\sinfo.*?task-internal.*?vmodl\.query\.PropertyCollector\.retrieveContents

[vmware_vpxd_null]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = ^\[?\d{4}-\d{2}-\d{2}[T\s][\d\:\.]{8,12}(?:[\+\-\s][\d\:]{5}|Z)?\s\[?\w+\s(verbose|trivia|info.*?task-internal.*?vmodl\.query\.PropertyCollector\.retrieveContents)

---

P.S.
I have successfully blocked vpxd-profil log with blacklisting it at inputs.conf.
But since inputs.conf is created automatically, controlling with nullQueue is wiser, I believe 🙂

Thank you.

[tomoyagoto]

follow-up to my own question.

I modified transforms.conf myself and now it works 🙂

I don't know what part of original conf prevented from exclusion.. but its ok

Splunk rocks!

---

excerpt of props.conf

[source::(?-i)...\\VMware VirtualCenter\\Logs\\vpxd-profiler-\d+.log(?:.\d+)?]
sourcetype = vmware:vclog:vpxd-profiler
TIME_PREFIX = \[
TIME_FORMAT = %Y-%m-%d %H:%M:%S.%3N
LINE_BREAKER = ([\r\n]+)
MAX_TIMESTAMP_LOOKAHEAD = 25
SHOULD_LINEMERGE = false
EXTRACT-extract_kv_pairs = (?<key>.+)[\s](?<value>[^\s]+)$
TRANSFORMS-null3 = vpxd_profiler_death

---

excerpt of transforms.conf

#NullQueues
[vpxd_profiler_death]
DEST_KEY = queue
FORMAT = nullQueue
REGEX = .

---

Thank you.