Getting Data In

Regex transform on a dynamic default host?

cphair
Builder

Hello,

I have Splunk on some Windows VMs that run on ESX hosts. For each guest, the ESX hosts generate customized performance data that I'd like to ingest into Splunk as well, in order to correlate the hosts' data with the data reported from Splunk within the guest OS.

The problem is that the custom performance data is saved under a different name than the guest OS, so I can't make a direct comparison if I just ingest the data--for instance, a guest OS might be named foo01456, but the ESX host knows it as bar01456, so Splunk would consider them two different machines. The machine name is not contained within the custom data, only in its filepath (e.g. \\dir1\dir2\bar01456\data.log). What are my options? It looks like editing inputs.conf to set the default host only will return me an exact match on a portion of the path, and I'd want to transform it from bar01456 to foo01456. Transforms.conf has a similar option, but I think the machine name has to be somewhere within the data for that to work. Do I understand this correctly? Is the name correlation better done at search time? Do I have to resort to a custom script?

http://docs.splunk.com/Documentation/Splunk/latest/Data/Setadefaulthostforaninput

Tags (2)
0 Karma

sowings
Splunk Employee
Splunk Employee

You can remap the host at indexing time by using props / transforms like this:

props.conf:

[<your_sourcetype>]
TRANSFORMS-hostname = esx_remap_host

transforms.conf:

[esx_remap_host]
# This corresponds to the full path to the logfile,
SOURCE_KEY = MetaData:Source
DEST_KEY = MetaData:Host
REGEX = /dir1/dir2/(.+)/data.log
FORMAT = host::$1

This, however, assumes a new hostname matched verbatim in the input filename. In your case, you might have to try something like:

[esx_remap_host]
REGEX = /dir1/dir2/prefix(\d+)/data.log
FORMAT = host::newprefix$1

I haven't tried this myself.

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...

Kubernetes Alerts That Work Out of the Box with AutoDetect detectors in Splunk ...

A deployment rolls out slowly, one pod sits not ready for a few minutes, and whoever is on call gets an alert ...

Observability at Your Fingertips Directly From Splunk Platform—for Free

At .conf, we announced our roadmap to bring observability foundations to Splunk Platform. Bringing metrics and ...