topic Send events from TA to different indexes depending on hostname in Getting Data In

Send events from TA to different indexes depending on hostname

torowa — Wed, 30 Sep 2020 01:13:32 GMT

Hi Splunkers.

What approach are people using to send events from a TA to different indexes depending on what the hostname?

For example:

We have the Splunk_TA_nix addon deployed out to our Linux machines.
We want the same source events pulled in for all of our Linux machines but need the events going to different indexes depending on the host name. For reasons of access/security we need the hosts sending to an index specific that hosts environment

i.e. for a given source:
- host1a, host1b and host1c sends events to "normal_index"
- host2a, host2b and host2c sends events to "secure_index"

Our goal here is to avoid having to maintain multiple Linux TAs, with the only difference being the "index = " line in the inputs.conf.
I realise this could be done by copying the TA to a different directory name and updating the index.conf in one of them to use a different index name but code inside the TA seems to expect the TA to sit in a directory of "TA_Splunk_nix" and would break if running inside a differently-named directory.

I don't want to manually change the directory name in the code as this makes upgrading the TA a nightmare.

I've seen something similar to this done using whitelists to detect a hostname in the directory of "monitor" stanzas but this doesn't look to be available for "script" stanzas.

What I am asking:
What method are people using to send events from a TA to different indexes depending on the hostname?

To clarify: I am not wanting to send an event to multiple indexes from the same host.
Thanks.

Re: Send events from TA to different indexes depending on hostname

whrg — Wed, 03 Jul 2019 06:54:26 GMT

I suggest you check out the documentation on "Filter event data and send to queues":

https://docs.splunk.com/Documentation/Splunk/7.3.0/Forwarding/Routeandfilterdatad#Filter_event_data_and_send_to_queues

Set your inputs.conf on all your Linux machines to "index = normal_index".

Now configure routing on your heavy forwarder (or on your indexer if you don't have a heavy forwarder) as follows. props.conf:

[host::host2a]
TRANSFORMS-index = set_secure_index
[host::host2b]
TRANSFORMS-index = set_secure_index
...

And transforms.com:

[set_secure_index]
REGEX = .
DEST_KEY = _MetaData:Index
FORMAT = secure_index

Re: Send events from TA to different indexes depending on hostname

astackpole — Wed, 25 Nov 2020 14:14:19 GMT

Were you able to get this problem resolved?

I am facing the same issue in my environment now and any guidance you may have would be appreciated!