I have a Linux forwarder running Splunk 4.1.2. This system uses TCP ports to listen for SYSLOG data from certain devices. When the log data comes in via these ports they are indexed nicely in Splunk. But the host value for these events are set to the Forwarder's host name. Is there a way I can REGEX the incoming SYSLOG information to grab the IP address near the beginning of the SYSLOG data, and set the host to this value?
Modifying the props.conf and transforms.conf on the Splunk Indexer should do the trick.
Configure a dynamically extracted host name for any source or sourcetype via transforms.conf and props.conf. Edit these files in $SPLUNK_HOME/etc/system/local/, or your own custom application directory in $SPLUNK_HOME/etc/apps/. For more information on configuration files in general, see "About configuration files" in this manual. Edits to transforms.conf
Add your custom stanza to $SPLUNK_HOME/etc/system/local/transforms.conf. Configure your stanza as follows:
[$UNIQUE_STANZA_NAME] DEST_KEY = MetaData:Host REGEX = $YOUR_REGEX FORMAT = host::$1
Fill in the stanza name and the regex fields with the correct values for your data.
Leave DEST_KEY = MetaData:Host to write a value to the host:: field. FORMAT = host::$1 writes the REGEX value into the host:: field.
Note: Name your stanza with a unique identifier (so it is not confused with a stanza in $SPLUNK_HOME/etc/system/default/transforms.conf). Edits to props.conf
Create a stanza in $SPLUNK_HOME/etc/system/local/props.conf to map the transforms.conf regex to the source type in props.conf.
$name is whatever unique identifier you want to give to your transform.
$UNIQUE_STANZA_NAME must match the stanza name of the transform you just created in transforms.conf.
Note: Optionally add any other valid attribute/value pairs from props.conf when defining your stanza. This assigns the attributes to the you have set. For example, if you have custom line-breaking rules to set for the same , append those attributes to your stanza.
Follow this link for some great examples:
So you would not add these settings to PROPS and TRANSFORMS on the system running Splunk Forwarder?
To test with I added the PROPS and TRANSFORMS to my Forwarder (not running light forwarder) and the host field did change correctly. Thanks for this information it was very helpful!