How to receive and index VMware logs using a Splunk 4.2.2 Windows universal forwarder and a Splunk 4.2.2 Linux indexer?

[05500]

*Environment
Index server: Splunk version is 4.2.2 on Linux
Forwarder: VMware with vCenter on Windows Server 2008 (Universal Forwarder is 4.2.2)

Question,
If we install a Universal Forwarder on this server (windows server 2008), what logs can we get?
Windows Event logs and performance logs only?

Actually if possible, we want to receive VMware logs, too.
How do I do this?

[fdi01 · Accepted solution]

If you install Universal Forwarder on this server (windows server 2008), you can get Windows Event logs; performance logs and data of Active Directory (Active Directory monitoring).

if you want to receive VMware logs, see following procedure below it can help you.

Configure Splunk App for VMware to receive syslog data
Prerequisites: Verifications
1. To configure ESXi log data collection, identify the machine to use as your data collection point. Verify that the ESXi hosts can forward data to that data collection point.
2. For the first installation, use an intermediate forwarder as your data collection point. Configure hosts to forward syslog data to the intermediate forwarder.

Step 1: Install a Splunk Universal Forwarder on your syslog server

Step 2: Create an inputs.conf file
Create an inputs.conf file in the system/local folder to monitor the ESXi hosts log files on the syslog server. Set the index and the source type before sending it to the intermediate forwarder.

1. For each monitor stanza in the inputs.conf file, specify the following settings:
   • sourcetype: vmw-syslog
   • index: vmware-esxilog. See "Configure your inputs" [monitor:///var/log/.../syslog.log] disabled = false index = vmware-esxilog sourcetype = vmw-syslog
2. Configure forwarding on your syslog server in outputs.conf to send data to your indexer or intermediate forwarder, which is the Splunk Enterprise instance on which Splunk_TA_esxilogs is installed.

Step 3: Install and configure Splunk_TA_esxilogs

Install and configure Splunk_TA_esxilogs on the machine that receives log data from your syslog server.
Install Splunk_TA_esxilogs under $SPLUNK_HOME/etc/apps. This technology add-on is included in Splunk App for VMware. It collects syslog data from the ESXi hosts and maps the data into the dashboards in Splunk App for VMware.

Step 4: Configure Splunk_TA_esxilogs

1. Assign the host field (on the machine where Splunk_TA_esxilogs is installed). The Splunk App for VMware can not determine the originating host for the data when you use a syslog server as your data store and you forward that data to the Splunk indexer.

2. Optionally create an index time extraction that takes the actual host name from the event that passes through, so that the log files can be associated with the correct host. By default, the host name is that of the syslog server. This step is not required when you use an intermediate forwarder, as the Splunk App for VMware automatically assigns the host based on the original data source.

3. Assign the host field. Create a local version of props.conf and transforms.conf in the $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/local/ directory and add the regular expressions to extract the host field.

In this example regular expression extraction in props.conf calls the set_host stanza of transforms.conf where the regular expression extraction extracts the host. The source and sourcetype fields are extracted by the settings in the props.conf and transforms.conf files in $SPLUNK_HOME/etc/apps/Splunk_TA_esxilogs/default. Do not override these fields in the local versions of these files.

Here's an example of the entry for props.conf:
[vmw-syslog]
……
TRANSFORMS-vmsysloghost = set_host

Here's the example for transforms.conf

[set_host]
REGEX = ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+([^ ]+)\s+)
DEST_KEY = MetaData:Host
FORMAT = host::$1

1. If the sourcetype is not correct, check the regular expressions in the stanzas [set_syslog_sourcetype] and [set_syslog_sourcetype_4x] in Splunk_TA_esxilogs/default/transforms.conf.

The following is an example of an entry in transforms.conf:

[set_syslog_sourcetype]
REGEX = ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+[^ ]+\s+)?([A-Za-z\-]+)(?:[^:]*)
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::vmware:esxlog:$1

Where:
- ^(?:\w{3}\s+\d+\s+[\d\:]{8}\s+[^ ]+\s+)? is used to extract the datetime field and host field.
- ([A-Za-z\-]+) is used to extract the sourcetype. and

- (?:[^:]*) defines the limit. sourcetype is followed by : or [ .

for more information go to this link:
http://docs.splunk.com/Documentation/VMW/3.1.4/Installation/CollectlogdatafromESXihosts#Configure_Sp...

or
http://docs.splunk.com/Documentation/VMW/3.1.4/Installation/CollectlogdatafromESXihosts#Use_an_inter...

[miteshvohra]

Splunk Enterprise 4.2.2 has reached EOL on Oct'2013. However, here is the link to getting data in using Splunk 4.2.2 : http://docs.splunk.com/Documentation/Splunk/4.2.2/Data/WhatSplunkcanmonitor.

Splunk App for VMware, being a premium app, will need at least Splunk Enterprise 6.0.6 or later version. Here is the link to know platform and hardware requirements: http://docs.splunk.com/Documentation/VMW/3.1.4/Installation/Platformandhardwarerequirements.

Hope this helps.

Mitesh.

[satishsdange]

Please refer to below site for the use cases for VMware environment - http://docs.splunk.com/Documentation/VMW/3.1.4/User/Commonusecases

You can also receive logs from ESX host using syslog.

http://docs.splunk.com/Documentation/VMW/3.1.4/Installation/CollectlogdatafromESXihosts