How to get VMware Per VM Log files into Splunk (vmware.log)?

[steubens]

Hi, can anyone tell us how to get "Per VM" log files into splunk. We already have esx syslog outs going to splunk as well as the vcenter log collector... but what I want to see in splunk for troubleshooting, is the contents of the log files that are produced by each VM inside its VMFS folder as it runs... the log file is called "vmware.log" and is rolled off to subsequent vmwware-n.log files every so often by the esx server. If w can get the live contents of vmware.log streaming into splunk just like syslog does for the host, that would be AWESOME!

thanks in advance.

[lguinn2]

If only there was a Splunk forwarder for ESXi! (Which VMware is unlikely to ever allow.) As sk314 suggests, you could use the API. It's not trivial, but you may be able to find some tutorials, etc. on the Internet.

Also, http://www.vmware.com/products/esxi-and-esx/management.html says "vSphere exposes logs from all system components using industry-standard syslog format, with the ability to send logs to a central logging server." However, the ESXi syslog only captures ESXi-level events. It looks like you are already doing this.

But this may work to add the vmware.log info to the ESXi syslog:

For each VM, edit the .vmx file setting as follows

vmx.log.destination = "syslog-and-disk"
Or do it via the advanced settings for a VM in the vSphere client. This should keep the normal vmware.log, but also write the events to the ESXi syslog.

Finally, you might want to take a look at Splunk's VMware app, but the app might be overkill if this is all that you want to do...

[splunkreal]

This works:

https://docs.splunk.com/Documentation/AddOns/released/VMW/VMwareAPI

    Navigate to your virtual machine vmx file.

    -> Add vmx.log.destination = "syslog-and-disk" to your virtual machine vmx file.

    -> Name your vm log entry. (Example:vmx.log.syslogID = vmx[splunkdata])

    Check the log entry in /var/log/syslog of your ESXi host to verify the syslog is being forwarded.

[sk314]

You could try using the vSphere SDK for this?