Upgraded Splunk app for VMware to 3.4.0 with VMware v6.5.0...
we are not seeing any sourcetype extractions based on props and transforms in Splunk_TA_vcenter...
Splunk_TA_vcenter is installed on HF(syslog), Indexer and SearchHead(stand alone search head, dedicated for vmware)
vCenter---->HF(syslog)------>Indexer------>SearchHead
#our custom inputs on HF
inputs.conf
[monitor:///var/log/vmware_hosts/vcenter-*.myorg/messages*]
disabled = 0
sourcetype = vclog
host_segment = 4
index = vmware-vclog
#props and transforms are from Splunk_TA_vCenter
props.conf
[vclog]
SHOULD_LINEMERGE = false
TRANSFORMS-vmwvclogsourcetype = set_vclog_sourcetype
transforms.conf
#Sourcetype Extraction
[set_vclog_sourcetype]
REGEX = ^([a-z\-]+)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::vmware:vclog:$1
I had a doh moment with this early on. I figured the props and tranform conf files were not the correct ones being loaded. Try using the btool to make sure your configuration are not being overwritten by locally deployed conf files
hi @mcnamara,
Can you please provide few details?
1) Provide Vcenter details
version :
OS: Windows / Linux /..
2) How you collecting VCenter vclog data on HF, Via monitoring stanza OR using syslog?
3)Have you configured custom syslog or log files? I found above-defined extraction will work for only Syslog data. The rex used for sourcetype extraction will work for Syslog event only.
4) Can you please check link and verify your steps?
Thanks
1) Provide Vcenter details
version :
OS: Windows / Linux /..
it's a linux appliance vCSA 6.5
2) How you collecting VCenter vclog data on HF, Via monitoring stanza OR using syslog?
vCenter--->syslog forwarding-udp/514(HF)-->using a monitoring stanza as defined above
3)Have you configured custom syslog or log files? I found above-defined extraction will work for only Syslog data. The rex used for sourcetype extraction will work for Syslog event only.
we are thinking to do a custom extraction for sourcetype
4) Can you please check link and verify your steps?
http://docs.splunk.com/Documentation/AddOns/released/VMW/vCenterlogs#Collect_vCenter_Server_Applianc...
we are forwarding vCenter logs to HF, monitoring using a custom stanza, we haven't made any tweaks based on this doc.
Then we have to change rex in transforms.conf.
can you please try below rex in set_vclog_sourcetype
stanza?
REGEX = ([a-z\-]+)\[
I have already changed the regex, and i am checking on the right field extractions based on props and transforms.
I'm having the same issue. I've made these REGEX changes. The only difference on our implementation, is that I am ingesting directly from syslog on TCP:1517. I have checked for local configurations, and while I have a local inputs.conf (that has the copied stanza from the setup instructions), I do NOT have a local transforms.conf.
I have vCenter Appliances running 6.5, and I'm currently running Splunk 6.6.4.
Has anyone found a solution for this issue? (by the way, I am both happy and sad that I'm not the only one having this issue)
Sincerely,
Matthew Granger
Can you post some sample events?
I have the sample events below, but according to the docs, the props/transforms should extract these sourcetypes
vmware:vclog:vpxd
vmware:vclog:vpxd-profiler
vmware:vclog:vpxd-alert
vmware:vclog
vmware:vclog:cim-diag
Nov 13 09:28:33 vc-mir01 vpxd[2860] Event [5092318] [1-1] [2017-11-20T15:28:33.26976Z] [vim.event.UserLogoutSessionEvent] [info] [AD\irkms-sme-daleapp] [] [5092318] [User AD\itcms-sme-daleapp@192.168.90.89 logged out (login time: Monday, 13 November, 2017 15:28:21, number of API invocations: 1, user agent: web-client/6.5.0)]
Nov 13 09:27:50 vc-mils vmon[1324] Executing op API_HEALTH on service netdumper...
Nov 13 09:27:50 vc-mils vmon[1324] Constructed command: /usr/bin/python /usr/lib/vmware-vmon/vmonApiHealthCmd.py -n vmware-vpostgres -f /dev/shm/vmware-postgres-health-status.xml