I hava a log on a Windows server like this: D:\SplunkTest\confidencial.log and on this log, I have data like this:
nome:user1,cpf:32861123800
nome:user1,cpf:32861123822
and I would like to mask all data after "cpf=" with XXXXX, so I configure /opt/splunk/etc/system/local/props.conf with script below:
[source::D:\\SplunkTest\\confidencial.log]
TRANSFORMS-anonymize = cpf-anonymize
root@LXPNHHSPLUNK01:/opt/splunk/etc/system/local#
and configure /opt/splunk/etc/system/local/transforms.conf with:
[cpf-anonymize]
REGEX = (?m)^(.*)cpf=\w+(\w{4}[&"].*)$
DEST_KEY = _raw
FORMAT = $1cpf=###########$2
However, after doing this configuration, I write in the log and nothing is masked. Could someone help me and point out where I'm going wrong?
Best Regards
You are taking the long way using the old approach; keep it simple like this in props.conf:
[confidencial-too_small]
SEDCMD-cpf_anonymizer = s/,cpf:\d+/,cpf:###########/
Deploy to the first full instance of splunk that handles the data (usually the Indexers but you might have HF) and restart all splunk instances there.
You are taking the long way using the old approach; keep it simple like this in props.conf:
[confidencial-too_small]
SEDCMD-cpf_anonymizer = s/,cpf:\d+/,cpf:###########/
Deploy to the first full instance of splunk that handles the data (usually the Indexers but you might have HF) and restart all splunk instances there.
I followed his suggestion however after having changed the props.conf restart the splunk but still did not work I write in the log file something like: "testing, cpf = 156766164343" and nothing is changed.
See the file: /opt/splunk/etc/system/local# cat props.conf
[sourcetype::confidencial-too_small]
SEDCMD-cpf_anonymizer = s/,cpf:\d+/,cpf:###########/
Your stanza header is wrong; remove the sourcetype::
as indicated in my updated answer.
Did this work for you, @justodaniel? Be sure to UpVote
any helpful answers and click Accept
if one worked.
Sorry I had not seen the answer. I'll do what you said and I'll tell you the result here.
Thank you! worked perfectly.
Hi,
your regex does not look like it is written for you raw data? Moreover you might want to put that into an extra app and not into system/local
.
Maybe something like this might go into your intended direction:
props.conf:
[source::D:\\SplunkTest\\confidencial.log]
TRANSFORMS-anonymize = cpf-anonymizer
transforms.conf:
[cpf-anonymizer]
REGEX = ^(.+?),cpf:.*$
FORMAT = $1,cpf:###########
DEST_KEY = _raw
Edit: simplified regex, added props.conf
and removed single quotes as mentioned by @ddrillic, thanks.
This will take everything after ,cpf:
until the end of the event and replace it by ###########
.
Hope that helps.
Hi @bjoernjensen,
Thanks for your help but I think that I still make something wrong. Look my files:
/opt/splunk/etc/system/local# cat props.conf
[source:D:\SplunkTest\confidencial.log]
TRANSFORMS-anonymize = cpf-anonymizer
/opt/splunk/etc/system/local# cat transforms.conf
[cpf-anonymizer]
REGEX = '^(.+)(,cpf:)(.+)$'
FORMAT = $1$2###########
DEST_KEY = _raw
/opt/splunk/etc/system/local#
Could you help me?
I believe it should read REGEX = ^(.+)(,cpf:)(.+)$
, meaning, no single quotes...
Not yet. Do I need to restart the service for the change to start working?
See the files:
/opt/splunk/etc/system/local# cat transforms.conf
[cpf-anonymizer]
REGEX = ^(.+)(,cpf:)(.+)$
FORMAT = $1$2###########
DEST_KEY = _raw
/opt/splunk/etc/system/local# cat props.conf
[source:D:\SplunkTest\confidencial.log]
TRANSFORMS-anonymize = cpf-anonymizer
/opt/splunk/etc/system/local#
In your props.conf
stanza you have to type double colon (::
) after source
:
[source::<SOME_SOURCE>]
I followed your instructions but it still did not work even after the service restart.
See conf files:
/opt/splunk/etc/system/local# cat props.conf
[sourcetype::confidencial-too_small]
TRANSFORMS-anonymize = cpf-anonymizer
/opt/splunk/etc/system/local# cat transforms.conf
[cpf-anonymizer]
REGEX = ^(.+)(,cpf:)(.+)$
FORMAT = $1$2###########
DEST_KEY = _raw
/opt/splunk/etc/system/local#
I changed source by sourcetype to see if there is anything related to sorce, however both one and the other do not mask the data
Where can I be wrong? What else could you do to solve it?
Great catch @bjoernjensen ; -)
Yes, and if you want to reindex, you have to set crcSalt ...
https://docs.splunk.com/Documentation/Splunk/latest/admin/Inputsconf