I have a log that look like this:
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
I want to remove all Deny(Eg: ORG PRINTER SELECT = Deny)
On my transforms.conf I have
[removedeny]
REGEX = ^([A-Za-z0-9\S\s]+\s=\sDeny,)$
FORMAT = $1$2
DEST_KEY = _raw
On my props.conf I have
REPORT-removedeny= removedeny
But it is still not working: Do I need to use the field name, or change my regex? am I applying the proper user of Transform?
Thank you,
Give this a try. No transforms.conf change needed.
props.conf
[YourSourceType]
..
Other configurations
..
SEDCMD-deny = s/(\[)*(\w+\s+)+=\sDeny(,|\s)//g
I tried with following sample data and below that is the outpt I received.
Sample data:
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny, SESSION CLEAN UP = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>TEST = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant ]
Output after SEDCMD:
<ReceivedPermissions>TEST = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant ]
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
Give this a try. No transforms.conf change needed.
props.conf
[YourSourceType]
..
Other configurations
..
SEDCMD-deny = s/(\[)*(\w+\s+)+=\sDeny(,|\s)//g
I tried with following sample data and below that is the outpt I received.
Sample data:
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny, SESSION CLEAN UP = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>TEST = [ EMULATEANOTHERUSER = Deny ], APPLICATION = [ PRIV FILE FDIC CAS = Deny, SESSION CLEAN UP = Deny, PRIV FILE IRS IBFD = Deny, WEB HOSTED CLIENTID IPAD = Grant ]
Output after SEDCMD:
<ReceivedPermissions>TEST = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant ]
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
<ReceivedPermissions>EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant,
For you first question, see this. Since you're doing multiple remove, SEDCMD is your guy.
http://answers.splunk.com/answers/9456/performance-difference-between-using-sedcmd-and-older-regextr....
Great job resolving the extra spaces issue. I was getting that too but somehow didn't show when pasted the result here.
we found the answer, we add and extra \s here sDeny(,\s|\s)
before
([)(\w+\s+)+=\sDeny(,|\s)
after
([)(\w+\s+)+=\sDeny(,\s|\s)
Thank you ,
Also, can this be test from the Splunk search page, so I can play with regex without being restarting splunk indexers
SEDCMD-deny = s/([)*(\w+\s+)+=\sDeny(,|\s)//g
Two Question: Do you happen to know if:
-As far as performance goes, is there any difference in change the Transforms.conf or, add only SEDCMD on props.conf ?
-Also, the white spaces are related to Splunk way to deal with the Sed, or the regex need to be tweaked?
“EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, PRACINSIGHTPHONE = Grant, DESKTOP PRACLAW CORP = Grant, KCALERT MONTHLY = Grant, COINV ALERTS = Grant, ANNOTATIONS = Grant, DESKTOP PRACLAW EMP = Grant, DESKTOP PRACLAW CAP = Grant, MYBI- BLC ZONE = Grant,
”
Thank you,
Glad it helped. Let me know if there are any followup questions, else just mark the question answered.
It worked great!
one thing though, I notice that your results did not have the big spaces that mine have. I think I might be able to fix that tweaking your regex. That is great man!
EMULATION = [ ], APPLICATION = [ WEB HOSTED CLIENTID IPAD = Grant, PRACINSIGHTPHONE = Grant, DESKTOP PRACLAW CORP = Grant, KCALERT MONTHLY = Grant, COINV ALERTS = Grant, ANNOTATIONS = Grant, DESKTOP PRACLAW EMP = Grant, DESKTOP PRACLAW CAP = Grant, MYBI- BLC ZONE = Grant, KEYCITE ALERTS = Grant, EMAIL DELIVERY = Grant, TAX KPMG USER = Grant,
I was trying this same property when I got your message,
it seems to be a much easier solution. I am testing your regex at this moment, it seems to be working much better then mine was, and yours is actually a lot simpler too.
As soon as I finish my test I will let you know the results,
Thanks a lot, you have been really helpful!
Yes, the granted are the only ones I want to see as result,
If we take this as sample log entry, what should be the expected output??
Input:
Output??
See these links for masking your data (based on regex)
http://answers.splunk.com/answers/62374/anonymize-the-sensitive-data-no-gaurantee-in-splunk
http://answers.splunk.com/answers/76825/anonymize-data-using-regex-transform