Clone and mask events

[krdo]

Hi,

We want to restrict access to some fields of an event (e.g. message and obviously _raw) which may contain sensitive information.

One approach is to send events which contain sensitive information to a special index (indexB) and restrict access to this index. The problem is that users without access to indexB don't even know that the event occurred and don't have access to the remaining fields.

Another approach would be to use data masking, but this would remove the information completely which we want to avoid (some users may and need to see the complete event).

We also tried using the search restriction * | fields - _raw message (does exactly what we need in normal searches) but is not supported by splunk.

Is there a way to restrict access to specific fields? Or is it possible to clone & mask events to a second index?

BTW, we use heavy forwarders & splunk 6.2.3

[mrgibbon · Accepted solution]

Best thing to do, IMHO:
Index sensitive data into its own index. Use RBAC to secure it.
Then use summary indexing to provide a source of information that doesn't contain the sensitive parts.
That way, you keep the sensitive stuff hidden, and still provide valuable info to those that need it.

If you still decide to mask in the end, info can be found here:
http://docs.splunk.com/Documentation/Splunk/6.2.3/Data/Anonymizedatausingconfigurationfiles

[krdo]

Thanks for the reply, looks like this is the only option. I'll check if summary indexing can be used in our scenario.

[krdo]

We now use two separate indexes (A and B). Events with sensitive data are re-routed to index B while all others go to index A. A saved search uses summary indexing to select all events from index B, removes fields with sensitive data and adds them to index A. A configuration stanza in props.conf ensures all fields are set to their original values, this way users can search for events as usual. Obviously summary indexing adds a slight delay (WCET 2 minutes in hour case).
Thanks for pointing me in the right direction!

[chimell]

Hi krdo

By running the query below you will not see _raw and message fields because the expression fields - _raw message exclude them in the list of fields.

   begin your search here| fields  - _raw message |table *

Upload tutorialdata and run the following query you will see that in the fields that will appear you will not have categoryId action _time clientip bytes fields.Just test

sourcetype=access_* |fields - _raw categoryId action _time clientip bytes|table *

[krdo]

yes, the fields command does exactly what we want, but it is not possible to use it in a search restriction AFAIK.