I want to do an automatic lookup from a CSV file on index time, and add new fields to the event. I got this working, but what if I want to anonymize the field used as lookup key afterwards?
Using this won't work since it seem to happen prior to the lookup runs:
props.conf
[default]
SEDCMD-anonymize = s/username=(......)/username=XXXXXX/g
Help appreciated!
It cannot be done without augmenting the data at Index-Time to include the lookup details. Lookups happen at Search-Time ALWAYS.
Hope Splunk enabled a similar option for "tokenisation" of certain fields at index time (eg credit card numbers for apple pay)
It cannot be done without augmenting the data at Index-Time to include the lookup details. Lookups happen at Search-Time ALWAYS.
Thank you for clarifying! So I need to populate the data prior to indexing, in order for this to work.
Yes, think of it this way: any field created at Index-Time
must be based off of a continuous string inside of the event itself (e.g. field X
starts as position Y
and ends at position Z
) or in the meta-data for the event (e.g. source
). This is how all Index-Time
fields are defined and there is not (and probably never will be) any exception. Once I realized this, my thinking about fields became much more clear.
Does no one have a solution or guidance to this? Help is very much appreciated!