How do I index a value thats not in the message?

[Marinus]

I'm currently collecting logs on a lightweight forwarder. I'm adding a special field to the messages which I'd like to extract index it for later use at search time to enrich events with a lookup.

I don't want to add the field into the message. Is there s simple way to get my field to the indexer?

My first plan was to concatenate it to the host meta key on the forwarder and then extract and index it on the indexer. The host key can then be cleaned up.

[forwarder]

host=xyz,special_key

[indexer]

host=xyz

special_field=special_key

[seach_time]

enrich using special_field

[Stephen_Sorkin · Accepted solution]

http://answers.splunk.com/questions/5571/way-to-insert-create-field-based-on-source

[Marinus]

It's fine to do it for a source, I'm doing it at the event level. Each event can have it's own unique key.

[dwaddle]

That is pretty much it - your data that is not in "_raw" becomes a new indexed field that is included in the index data when the events are indexed. The caveat is these are index-time and cannot be changed/updated/removed without dumping and reindexing data. We use this in my shop today to tag data as coming from "test" versus "prod" for files on the same source path.

[Marinus]

Thanks Stephen, I'm just not sure that I'm following your logic here. Step 1, would be to locate my special value V and index it using a new meta key K::V
Step 2, would be to remove the value from the host
Am I missing something from your post?