Is there a way to encrypt sensitive data in index time and decrypt it in search time in Splunk?

[human96]

Hi folks,

Is there a way to encrypt sensitive data in index time and decrypt it in search time in Splunk ?
if yes, how can we do this ?

[mrmusso]

Hello, I also want to encrypt personal data in the collected data at index time and decrypt it at search time, just like human96.

It seems that implementing decryption at search time can be done with a custom command, but I'm currently researching and contemplating how to encrypt a specific field at index time.

Have you implemented a method to encrypt a specific field at index time?

Your insights would be greatly appreciated.

[PickleRick]

Remember that during the ingestion phase Splunk mostly processes the event as a whole - extractions (unless you have indexed fields) are done in search time.

So if you wanted to encrypt part of the raw message (for now even leaving aside the question how to do it), you'd have to extract a part of the message into a field, encrypt this field, replace the original part of the raw message with the encrypted field value and finally "forget" the extracted and encrypted field values (so they do not get indexed alongside the raw event). Very, very ugly and error-prone. And we haven't even touched the question about _how_ to encrypt the value.

[Chiranjeev]

Basically what you can do is use an algorithm or a custom function to encrypt part of the data, now to decrypt you can create a custom command that would call the decryption key and give you the expected output for it .this solution can have performance impacts on large sets of data as preprocessing may take time before data gets indexed.

[PickleRick]

One can think of a way to do something like that (probably depending heavily on external stuff like modular input encrypting the data before ingestion and external lookup decrypting it). But there are questions which you need to ask yourself first:

1) What's the use case. What are you trying to protect yourself from. I know it's a typical temptation to "do encryption so it's safe" but "security" implemented blindly ends up only being a burden and nuisance and often not adding any real protection. Especially if you don't know what you're protecting yourself from.

2) If you want to just encrypt data at rest (like storing your indexes on an encrypted filesystem) - are you OK with the performance penalty?

3) If you want to just encrypt some parts of the data (like in the form I described above), are you aware that you're losing ability to search on encrypted fields? (because splunk doesn't know its unencrypted contents and doesn't have indexes of them). And are you OK with the security implications of given decryption process?

There are so many questions. Encryption as such is not a goal. It's a tool.

[human96]

We are implementing an use case for a Financial institution. The requirement is the Credit card information should be encrypted in indextime and then the privileged role user can see the credit card information if needed in the search time.

[PickleRick]

OK. To make a long story short - Splunk natively doesn't support anything like this.

You would have to implement it manually.

And with that comes a great deal of potential problems and architectural decisions you'd have to make. Both about the process as a whole but also as some technical implementation details (which are connected to one another).

EDIT: There are some third-party apps on splunkbase which try to implement something like that but I wouldn't trust them for production use without further reviewing and tailoring to own needs.

[human96]

I recently came across the below link, seems to be very interesting. Do you have any idea about it ?
FINAL_FN120332_AngeloBrancato&DirkNitschke_Splunk_DataObfuscation

[gcusello]

Hi @human96,

have you seen this: https://www.splunk.com/en_us/blog/learn/end-to-end-encryption.html?locale=en_us ?

Ciao.

Giuseppe