topic Re: How to forward events to different indexes based on universal forwarder IP address? in Getting Data In

How to forward events to different indexes based on universal forwarder IP address?

securityninja — Sun, 14 Jun 2015 14:43:53 GMT

Hi there,

I have dozens of devices forwarding data through universal forwarder to a heavy forwarder, which in turn forwards data to a group of indexers.
Due to access provisioning demands, I would like data from each set of these devices to be indexed under a specific index, so users can be granted access to the specific indexes.

By specifying in props.conf and inputs.conf on the heavy forwarder, is it possible to achieve this result?

Any help would be much appreciated.
Splunk newbie.

Re: How to forward events to different indexes based on universal forwarder IP address?

acharlieh — Sun, 14 Jun 2015 17:01:28 GMT

If you have control over the UF configuration, (either directly, or through a configuration management solution or a Deployment Server) then you can set index in inputs.conf on the UFs and you're all set.

Alternatively, it would take some processing on the HF, you would need some common field to match on the HF (host might be a good one so long as your UF isn't setting host from another source other than the UF itself) so your props.conf on the HF could have stanza(s) like:

[host::hostnamepattern]
TRANSFORMS = rewrite-index

which of course references corresponding transforms.conf entr(ies) like:

[rewrite-index]
SOURCE_KEY = MetaData:Host
REGEX = (regex match against SOURCE_KEY value)
DEST_KEY = _MetaData:Index
FORMAT = targetindex (could use capturing groups from the regex here)

There's a lot of variability here of course, and if you're using INDEXED_EXTRACTIONS on your UFs, this method of HF configuration wouldn't work (since the UFs are sending fully parsed events in that case).

Re: How to forward events to different indexes based on universal forwarder IP address?

securityninja — Mon, 28 Sep 2020 20:16:15 GMT

Hi,
Thank you very much for your detailed answer.
Here are the changes I have done.

/opt/splunk/etc/system/local/props.conf:
[host::SALSABDCO101]
TRANSFORMS-dc_mog_qatar_cn = dc_mog_qatar_tn

/opt/splunk/etc/system/local/transforms.conf:
[dc_mog_qatar_tn]
REGEX = .*
DEST_KEY = _MetaData:Index
FORMAT = idx_dc_do_mog

These changes were done on both the indexers which are involved in indexing events from the host in question.
SALSABDCO101 is the host running the UF whose events I would like to be indexed into idx_dc_do_mog

Am I doing something incorrect, because I see events from host SALSABDCO101 being indexed into the index defined in inputs.conf under the domain controller app on the heavy forwarder.