Problem: Unable to send cooked data to two different Indexer ports

[splunk_kk]

Hello Experts,

I have an issue where I am unable to send cooked data to two different Indexer ports.

My flow of traffic is UF > HF > IDX

UF IP: a.a.a.a
HF IP: y.y.y.y
IDX IP: x.x.x.x

1) Universal Forwarder is monitoring more than one input with different sourcetypes.
2) My Indexer is listening on TCP 9997 and TCP 9996 (Same Indexer is listening on both the ports)
3) I want the data with sourcetype ABC to hit TCP 9996 and all other data to hit TCP 9997 on the Indexer.
4) When hitting TCP 9996, the index should change to pqr index but while hitting TCP 9997 the index should remain the same as mentioned in the monitor stanza in UF

My configuration is below:

UF config

inputs.conf

[monitor:///xyz/urgent]
index= my_index
sourcetype=ABC
disabled=false

[monitor:///var/important]
index= my_index
sourcetype=DEF
disabled=false

outputs.conf

[tcpout]
useACK=true
maxQueueSize=auto
readTimeout=300

[tcpout:HF]
server = y.y.y.y:9997

HF config

inputs.conf

[default]
host = def.sdf.ppp

[splunktcp://9997]

props.conf

[ABC]
TRANSFORMS-routing = route_data

transforms.conf

[route_data]
REGEX = .
DEST_KEY = _TCP_ROUTING
FORMAT = urgent

outputs.conf

[tcpout]
defaultGroup = important
maxQueueSize = 50MB
useACK = true

[tcpout:important]
server = x.x.x.x:9997

[tcpout:urgent]
server = x.x.x.x:9996

Indexer config

inputs.conf

[default]
host = abcdef.xyz.ppp

[splunktcp://9997]

[splunktcp://9996]
index = pqr

This is not working for me. The sourcetype ABC is still sending data on TCP 9997 and not changing the index.

Have I missed something? is there anything wrong with the config?

[esix_splunk]

Think about your issue, I think you would be better to approach this differently. Instead of trying to assign the index or priority based on the inbound SplunkTCP port, why dont you add metadata at the ingest time. Add a metadata field such as PRIORITY-HIGH, and then at the HF layer, filter that to a specific index and all other data to general indexes...

See documentation here : http://docs.splunk.com/Documentation/Splunk/6.5.0/Data/Configureindex-timefieldextraction

Note, this does add an indexed field, which does effect indexing performance. But this is a better approach then trying to work with multiple TCP inputs and trasnforms.

[splunk_kk]

Thanks Esix, That worked for me.... 🙂 ok a quick question, If in case I need to send the data from sourcetype ABC to both the ports (9997 and 9996) and change the index when it hits 9996. will that be possible? if I change the index metadata at HF it will change the index for both the ports.

Do I need to write props and transforms in Index for that to work? I doubt

[esix_splunk]

Its not really possible to do this at the indexer side. Basically Splunk isnt able to change the meta data on multiple listening ports since it doesnt track which input port the data comes in on.

[esix_splunk · Accepted solution]

In your inputs.conf on the indexers, for splunktcp, index = is not a valid option.

You cannot do it this way. You would need to rewrite the index metadata for the sourcetype on your HF by adding another transform and change the index metadata.