How to direct incoming data from heavy forwarder to index by host name?

[jgorman_THG]

Hi,

I have data coming in from multiple hosts using either syslog, or a universal forwarder, going into 3 heavy forwarders, and then forwarding to SplunkCloud.

I've created 3 indexes - Financial, Infrastructure, and Security - and I would like to separate the data by host name

So I want data from "financial_server1" to go to the "financial" index, and data from "Firewall_1" to go to the "Security" index.

Can someone give me an example of how this would be done?

Thanks,

JG

[somesoni2]

This is how you can override the index name based on a sourcetype (sourcetype used here is "mysourcetype"). You can configure this based on host name as well by replacing "mysourcetype" in props.conf with "host::YourHostName".
On your indexer or heavy forwarder:
etc/system/local/transforms.conf

 [overrideindex]
 DEST_KEY =_MetaData:Index
 REGEX = .
 FORMAT = my_new_index

etc/system/local/props.conf

 [mysourcetype]
 TRANSFORMS-index = overrideindex

[jgorman_THG]

Ok Cool.

So if I wanted "host1" to go to index "Infrastructure" I would do

Transforms.conf:

[overrideindex]
DEST_KEY=_MetaData:index
REGEX = .
FORMAT = Infrastructure

Props.conf:

[host::host1]
TRANSFORMS-index = overrideindex

Correct?

Do I need to restart splunk after making this change? Will I need to do seperate entries for each host, or is there a way where I can enter all the applicable hosts in the same [host::***] line?

Since I have multiple indexes, in transforms would I put for example [overrideindex] [overrideindex1] [overrideindex2]?

Also does this also work for wineventlog data?

Thanks a lot! I really appreciate the help.

JG

[somesoni2]

The configuration looks correct (for moving data from host1 to Infrastructure index).
Yes, you would need to restart your HF after making this change.
You can re-use the entry in transforms.conf for each host that you need to more to same index.
You need to define separate transforms.conf stanza for each index.
It'll work for any type of data coming to HF (from UF OR syslog).

[jgorman_THG]

HI

So I made the following changes in the etc/system/local on all 3 heavy forwarders, I then ran on all 3 heavy forwarders:

| extract reload=T

This had no effect so I then restarted each one and still there was no change.

The heavy forwarders are feeding into SplunkCloud. The security index exists on splunk cloud. does it need to be on the heavy forwarders as well?

What am I doing wrong? Should this not be sending events from xx.xx.x.xx to index "security" in SplunkCloud?

Transforms.conf

[redirect_to_security]
DEST_KEY = _MetaData:Index
Regex = .
FORMAT = security

Props:

[host::xx\.xx\.x\.xx]
TRANSFORMS-index = redirect_to_security

[somesoni2]

It should. You would need to restart Splunk services on Heavy forwarder for these changes to take effect (bin/splunk restart).

[jgorman_THG]

So strange. I restarted all the heavy forwarders and still have the same result. There's nothing special that needs to be done since it is forwarding to SplunkCloud?

Thanks

[jgorman_THG]

Yes! Splunk recognizes and sets the host for each device.

Thanks

[dshpritz]

Are the items that you want to separate the data by available in the data? That is, are the hostnames in the eventdata itself?