I am collecting Windows Event Logs via either a Windows Event Collector or directly from each Windows server (with a Splunk UF installed). I am then using an intermediate forwarder to send to our indexers. This is due to network setup and location of the servers.
I have custom metadata that the intermediate forwarder is adding to identify where the logs are coming from and use in the index name.
The problem I am having is that the Windows Event Logs are hitting the Main index and not the one I've specified in transforms.
On the intermediate forwarder I have:
inputs.conf
[splunktcp:9997]
acceptFrom = *
On the destination Indexer I have:
props.conf
[source::WinEventLog:ForwardedEvents]
TRANSFORMS-Host = Set-Host-ComputerName
TRANSFORMS-LogName = Set-Sourcetype-LogName
TRANSFORMS-Index-Windows = Set-Index-Windows
[sourcetype::WinEventLog:*]
TRANSFORMS-Index-Windows = Set-Index-Windows
...
transforms.conf
[Set-Host-ComputerName]
REGEX = (?m)ComputerName=(.+)?\b
DEST_KEY = MetaData:Host
FORMAT = host::$1
[Set-Sourcetype-LogName]
REGEX = (?m)LogName=(.+)?\b
DEST_KEY = MetaData:Sourcetype
FORMAT = sourcetype::windows:event:$1
[Set-Index-Windows]
SOURCE_KEY = MetaData:customfield
REGEX = (.+)
DEST_KEY = _MetaData:Index
FORMAT = index-$1-win-event
If I understood your problem correctly, you want to put data into index named index-$custom$-win-event, I think you need to update your tranforms.conf stanza as follows:
[Set-Index-Windows]
SOURCE_KEY = field:customfield
REGEX = (.+)
DEST_KEY = _MetaData:Index
FORMAT = index-$1-win-event
The custom field is set like this on the intermediate forwarder:
[default]
_meta = customfield::ACustomString
If the custom field is set on the intermediate forwarder, I have to assume the intermediate forwarder is a heavy forwarder. If that's the case, it's sending cooked data to your indexer. The indexer won't re-cook data (under normal configuration, and best practice), so changing sourcetype/index/etc won't have any affect.
For a potential (against best practice) workaround, see this previous answer:
Reparsing cooked data coming from a heavy forwarder. Possible?
The Intermediate Forward is a Universal Forwarder. The custom metadata is being added to the logs.
SOURCE_KEY = field:customfield - This didn't work I'm afraid.