Splunk Search

Anonymize data in search-time

gofrolist
New Member

Hello,

I need to anonimize data in search-time and count by message.

Example.

source log file contains:

E 120327 153238.682 THREAD/CLASS - java.lang.InterruptedException A#1234
    at com.applyRequests(MobileSession.java:195)
Caused by: java.lang.InterruptedException
    at com.applyRequests(MobileSession.java:159)
E 120327 153239.682 THREAD/CLASS - java.lang.InterruptedException A#2345
    at com.applyRequests(MobileSession.java:195)
Caused by: java.lang.InterruptedException
    at com.applyRequests(MobileSession.java:159)
E 120327 153240.682 THREAD/CLASS - java.lang.AnotherException A#2345

in "splunk>search>event list" i want to see:

E 120327 153238.682 THREAD/CLASS - java.lang.InterruptedException A#XXXX
E 120327 153239.682 THREAD/CLASS - java.lang.InterruptedException A#XXXX
E 120327 153240.682 THREAD/CLASS - java.lang.AnotherException A#XXXX

in "splunk>search>table" i want to see:

message                               count
java.lang.InterruptedException A#XXXX 2
java.lang.AnotherException A#XXXX     1

but i still need to view a source log file with full stacktrace of exception

because of this i cant anonimize data A#2345 -> A#XXXX at index-time

Tags (2)
0 Karma
1 Solution

Jason
Motivator

You can use the rex command in sed-mode for this. Something like this should work:

... | rex mode=sed "s/Exception A#\d+/Exception A#XXXXX/g" 

By default, the command will work on the _raw field (the original text of the event) and will show the Xs in the main search view.

If you are using a field that is extracted already (for the table), use rex mode=sed field=myfieldname ... - you'll need multiple rex statements for multiple fields. If you're extracting the fields on the search bar, do the rex before you do the field extraction.

View solution in original post

Jason
Motivator

You can use the rex command in sed-mode for this. Something like this should work:

... | rex mode=sed "s/Exception A#\d+/Exception A#XXXXX/g" 

By default, the command will work on the _raw field (the original text of the event) and will show the Xs in the main search view.

If you are using a field that is extracted already (for the table), use rex mode=sed field=myfieldname ... - you'll need multiple rex statements for multiple fields. If you're extracting the fields on the search bar, do the rex before you do the field extraction.

Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...