Splunk Search

Transforms.conf - Hide values or make them anonymous

Path Finder

I have a log that look like this:

<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,

I want to remove all Deny(Eg: ORG PRINTER SELECT = Deny)

On my transforms.conf I have

[removedeny]
REGEX = ^([A-Za-z0-9\S\s]+\s=\sDeny,)$
FORMAT = $1$2
DEST_KEY = _raw

On my props.conf I have

REPORT-removedeny= removedeny

But it is still not working: Do I need to use the field name, or change my regex? am I applying the proper user of Transform?

Thank you,

Tags (2)
1 Solution

Revered Legend

Give this a try. No transforms.conf change needed.

props.conf

[YourSourceType]
..
Other configurations
..
SEDCMD-deny = s/(\[)*(\w+\s+)+=\sDeny(,|\s)//g

I tried with following sample data and below that is the outpt I received.
Sample data:

<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny, SESSION CLEAN UP = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>TEST = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant ]

Output after SEDCMD:

<ReceivedPermissions>TEST = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant ]
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, 

View solution in original post

Revered Legend

Give this a try. No transforms.conf change needed.

props.conf

[YourSourceType]
..
Other configurations
..
SEDCMD-deny = s/(\[)*(\w+\s+)+=\sDeny(,|\s)//g

I tried with following sample data and below that is the outpt I received.
Sample data:

<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny, SESSION CLEAN UP = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>TEST = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant ]

Output after SEDCMD:

<ReceivedPermissions>TEST = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant ]
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, 

View solution in original post

Revered Legend

For you first question, see this. Since you're doing multiple remove, SEDCMD is your guy.
http://answers.splunk.com/answers/9456/performance-difference-between-using-sedcmd-and-older-regextr....

Great job resolving the extra spaces issue. I was getting that too but somehow didn't show when pasted the result here.

0 Karma

Path Finder

we found the answer, we add and extra \s here sDeny(,\s|\s)
before
([)(\w+\s+)+=\sDeny(,|\s)
after
([)
(\w+\s+)+=\sDeny(,\s|\s)
Thank you ,

Path Finder

Also, can this be test from the Splunk search page, so I can play with regex without being restarting splunk indexers
SEDCMD-deny = s/([)*(\w+\s+)+=\sDeny(,|\s)//g

0 Karma

Path Finder

Two Question: Do you happen to know if:
-As far as performance goes, is there any difference in change the Transforms.conf or, add only SEDCMD on props.conf ?
-Also, the white spaces are related to Splunk way to deal with the Sed, or the regex need to be tweaked?

“EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, PRACINSIGHTPHONE = Grant, DESKTOP PRACLAW CORP = Grant, KCALERT MONTHLY = Grant, COINV ALERTS = Grant, ANNOTATIONS = Grant, DESKTOP PRACLAW EMP = Grant, DESKTOP PRACLAW CAP = Grant, MYBI- BLC ZONE = Grant,


Thank you,

0 Karma

Revered Legend

Glad it helped. Let me know if there are any followup questions, else just mark the question answered.

0 Karma

Path Finder

It worked great!

one thing though, I notice that your results did not have the big spaces that mine have. I think I might be able to fix that tweaking your regex. That is great man!

EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, PRACINSIGHTPHONE = Grant, DESKTOP PRACLAW CORP = Grant, KCALERT MONTHLY = Grant, COINV ALERTS = Grant, ANNOTATIONS = Grant, DESKTOP PRACLAW EMP = Grant, DESKTOP PRACLAW CAP = Grant, MYBI- BLC ZONE = Grant, KEYCITE ALERTS = Grant, EMAIL DELIVERY = Grant, TAX KPMG USER = Grant,

Path Finder

I was trying this same property when I got your message,
it seems to be a much easier solution. I am testing your regex at this moment, it seems to be working much better then mine was, and yours is actually a lot simpler too.
As soon as I finish my test I will let you know the results,

Thanks a lot, you have been really helpful!

Path Finder

Yes, the granted are the only ones I want to see as result,

0 Karma

Revered Legend

If we take this as sample log entry, what should be the expected output??
Input:
EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,

Output??
EMULATION WEB HOSTED CLIENTID IPAD = Grant,

0 Karma

Revered Legend
0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!