Removing fields from _raw or similar

[skippylou]

I'm trying to rex out a chunk of events, then remove that field from the events prior to piping to the cluster command.

So something similar to:

blah | rex "resolving '(?<some_fqdn>[\w\d\.]+)' " | fields - some_fqdn | cluster

So that the chunk extracted in the some_fqdn field doesn't contribute to making the event seem more unique.

I know that cluster by default analyzes the _raw field and am assuming that the 'fields - some_fqdn' is not removing from the _raw field.

So I guess my question is how do you remove a field from _raw prior to sending to another piped command, or how do you create a new field that is the equivalent of _raw minus some_fqdn (to then tell cluster to work off that new field)?

Thanks,

Scott

[ftk · Accepted solution]

You should be able to use rex in sed mode to redact the _raw field. Something like this:

blah | rex mode=sed "resolving 's/[\w\d\.]+//g' | cluster blahblahblah

[skippylou]

Worked perfectly! Thanks ftk!