Anonymize data in search-time

[gofrolist]

Hello,

I need to anonimize data in search-time and count by message.

Example.

source log file contains:

E 120327 153238.682 THREAD/CLASS - java.lang.InterruptedException A#1234
    at com.applyRequests(MobileSession.java:195)
Caused by: java.lang.InterruptedException
    at com.applyRequests(MobileSession.java:159)
E 120327 153239.682 THREAD/CLASS - java.lang.InterruptedException A#2345
    at com.applyRequests(MobileSession.java:195)
Caused by: java.lang.InterruptedException
    at com.applyRequests(MobileSession.java:159)
E 120327 153240.682 THREAD/CLASS - java.lang.AnotherException A#2345

in "splunk>search>event list" i want to see:

E 120327 153238.682 THREAD/CLASS - java.lang.InterruptedException A#XXXX
E 120327 153239.682 THREAD/CLASS - java.lang.InterruptedException A#XXXX
E 120327 153240.682 THREAD/CLASS - java.lang.AnotherException A#XXXX

in "splunk>search>table" i want to see:

message                               count
java.lang.InterruptedException A#XXXX 2
java.lang.AnotherException A#XXXX     1

but i still need to view a source log file with full stacktrace of exception

because of this i cant anonimize data A#2345 -> A#XXXX at index-time

[Jason · Accepted solution]

You can use the rex command in sed-mode for this. Something like this should work:

... | rex mode=sed "s/Exception A#\d+/Exception A#XXXXX/g" 

By default, the command will work on the _raw field (the original text of the event) and will show the Xs in the main search view.

If you are using a field that is extracted already (for the table), use rex mode=sed field=myfieldname ... - you'll need multiple rex statements for multiple fields. If you're extracting the fields on the search bar, do the rex before you do the field extraction.