Security

How can I decrypt log events after forwarder sends them but before indexing

dsharp1970
Engager

To meet an internal security requirement I must encrypt data at rest in some locations. I'd like this data in Splunk but must obviously decrypt it first. I see three possibilities.

1) Decrypt before, or as, the universal forwarder sends the data to the indexer.

2) Interrupt the data flow and decrypt after the forwarder sends the data but before indexing.

3) Let the encrypted data be indexed and then decrypt at search time.

The first has an obvious issue in that it requires the decrypt key be on, or accessible from, the 'secure' location and mostly defeats having the data encrypted to begin with. It would seem the third option would create a lot of extra work on the search heads and there will be hundred of millions of these log entires that would greatly compound the issue.

The best option would seem to be the second but I don't see any way to interrupt the data flow. I know there are sed scripts that I can call using config in props.conf that doesn't seem flexible enough to solve this. Any one have a clever way of solving this problem?

Tags (2)
0 Karma

martin_mueller
SplunkTrust
SplunkTrust

You might make option two work with a bit (a lot) of routing trickery.

Have the forwarders send the events with some encrypted payload to the indexers, using a sourcetype "foo-encrypted".
Set up routing for such sourcetypes to take an exit out of Splunk's index queue before the actual indexing, for example syslogout.
Send those events to a "decryption daemon" on your indexers that listens to the events routed off from the index queue and decrypts them.
Have the "decryption daemon" send the clear-text events back to Splunk, using a sourcetype "foo" that now gets sent along the regular indexing route.

Note, this a rough back-of-a-napkin draft... to actually implement this there surely is some more thinking and tinkering to be done.

0 Karma

Ayn
Legend

No, you cannot do this using option 2. Splunk has only very basic logic for transforming events based on regular expressions before indexing, nothing more.

0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...