topic How do you redact data that's already indexed? in Splunk Dev

How do you redact data that's already indexed?

lostbeatnik01 — Tue, 22 Jan 2019 14:07:42 GMT

We currently index logs into index=indexY at a rate of 2G – 5G a day with the retention set to 12 months.

One day last week, a code update started logging the user and password into the log in the form of ‘password=changeme’. Our Splunk environment is certified non-pci/non-private data and needs to remain so. We need to retain the event data due to security best practices, it is the event for access to a critical services, however the field needs to be redacted.

What are the best options for retaining the data while scrubbing the field? For instance 'password=change' should be redacted to 'password=SCRUBBED' or removed entirely. Please consider that that data at rest cannot contain confidential data, so masking it at the Search Head is not a viable solution.

Re: How do you redact data that's already indexed?

dkeck — Tue, 22 Jan 2019 17:30:20 GMT

there is a new function called ingest_eval. This lets you mask data at index time. I think its included since 7.2

https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/IngestEval

Re: How do you redact data that's already indexed?

harsmarvania57 — Wed, 23 Jan 2019 10:18:13 GMT

Hi @lostbeatnik01,

You can't alter indexed data so only option is to remove those data, you can use | delete command which hides data from searching but it will not remove data from server and to use this command you need can_delete capability in role, even admin role does not have this capability by default.

To anonymize any new data you can use SEDCMD- or combination of props and transforms, have a look at document https://docs.splunk.com/Documentation/Splunk/7.2.3/Data/Anonymizedata