Hi,
I'm wondering if there is a way to prevent a sensitive key-value pair that exists in cs_Cookie from appearing in Splunk. I have tried using SEDCMD on the forwarder, and it does change the _raw data, but the indexed value of cs_Cookie still contains the original data. For example:
IIS log
.. cs_Cookie ..
.. foo=bar;hide=me ..
props.conf
SEDCMD-cookie-cleaner = s/hide=\w+/hide=XXXX/g
As expected, this changes the _raw data to:
.. foo=bar;hide=XXXX ..
But, when I expand an event:
cs_Cookie="foo=bar;hide=me"
How is the original value making it to the indexer, and how can I get rid of it?
Thanks!
You have done using sed script in props.conf. As per Splunk documentation sed scripts act only on _raw field.
try anonymyzing data using regex transform and using the transform in props.conf
For more details, check this
http://docs.splunk.com/Documentation/Splunk/6.1.2/Data/Anonymizedatausingconfigurationfiles
For the sample log that you have given, the foo=bar is in 11th position. I am taking space as the separator between positions.
For your log sample, please see below.
transforms.conf
[anonymize_IP_Address]
REGEX = (?i)^(?:[^ ]+ ){10}(?:foo=)([^ ]+)
DEST_KEY = _raw
FORMAT = $1###$2
props.conf
[my_sourcetype]
TRANSFORMS-include = anonymize_IP_Address
Since splunk automatically extracts the key value pairs and if the above configuration doesn't work then you add KV_MODE = none to your sourcetype in props.conf. For more information on KV_MODE, see props.conf splunk documentation
Here is a sample:
#Fields: date time s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Cookie) cs(Referer) sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
2014-07-30 22:25:39 ::1 GET / test16 8080 - ::1 Mozilla/5.0+(Windows+NT+6.1;+WOW64;+Trident/7.0;+rv:11.0)+like+Gecko foo=bar - 200 0 0 299 281 0
I have simplified it, but basically I would want:
cs(Cookie)="foo=bar"
to be transformed into
cs(Cookie)="foo=###"
I had to anonymize a field in my log events.. and i did this and it worked
transforms.conf
[anonymize_IP_Address]
REGEX =
DEST_KEY = _raw
FORMAT = $1####$4
props.conf
[my_sourcetype]
TRANSFORMS-include = anonymize_IP_Address
If you can post your log events, then we can help
Thanks for you quick reply.
I have already tried to use a transform unsuccessfully.
Following an example I tried
SOURCE_KEY = MetaData:cs_Cookie
DEST_KEY = MetaData:cs_Cookie
but I still get he original values. Is this the right way to anonymize an extracted field?