Adding Index-time field on intermediate forwarder

[greich]

I need to trace the data from the originating forwarder through intermediate forwarders or directly onto indexers. I am trying to add a field when the data goes through the intermediate forwarder.

Configuration

Data collection by Heavy Forwarders

inputs.conf
[default]
host = shorthostname (of the HF)

sending all to Intermediate Forwarders

inputs.conf
[default]
host = shorthostname (IF)
router = shorthostname (IF)

props.conf
[default]
TRANSFORMS-router = addrouter

transforms.conf
[addrouter]
SOURCE_KEY = router
REGEX = (.*)
FORMAT = router::$1
WRITE_META = true

[accepted_keys]
DCIF_NAME = router

And finally on the search head
fields.conf
[router]
INDEXED = true

It seems that:
1- everything that is sourced directly on the IF (syslog and splunk logs) has the field "router"
2- everything incoming on TCP from HF does not have the field "router"

What I am missing to mark incoming cooked data?

[jacobwilkins · Accepted solution]

This is expected behavior. You are going to have to re-parse your cooked data:

See here:
http://answers.splunk.com/answers/97918/reparsing-cooked-data-coming-from-a-heavy-forwarder-possible...

On your IF (which probably needs to be a HWF), do this in inputs.conf:

[splunktcp]
route=has_key:_utf8:parsingQueue;has_key:_linebreaker:parsingQueue;absent_key:_utf8:parsingQueue;absent_key:_linebreaker:parsingQueue

[greich]

Yes IF are HF. On the first instance, your solution seems to work perfectly. Will let soak, and confirm later. Thank you.

[greich]

that did it, with no noticeable impact on CPU on the IF (was expecting some). Thanks again

[greich]

I must mention that a start message:
Checking conf files for problems...
Invalid key in stanza [default] in /opt/splunk/etc/system/local/inputs.conf, line 3: router (value: blahhost)

[greich]

adding the key to the .conf.spec should resolve this