Splunk HEC Obfuscate data

[lufermalgo]

Hi community,

I need your help to resolve a question. Is it possible to obfuscate / mask data that is sent via HEC?

Please can you give me an example.

Thank you.

[Sukisen1981]

The HEC operates in any other way that other indexes do. There are 2 possible solutions
1) And I like this more - obfuscate your data in the source application, you might be using java/python/.net ALL of them have data masking functions
2) The HEC index and source/sourcetypes are just like any other data source, you can go to the .conf files , use the HEC .When you send the data and if you configure something like this
{
"time": 1426279439, // epoch time
"host": "localhost",
"source": "datasource",
"sourcetype": "txt",
"index": "main",
"event": { "Hello world!" }
}
You can then just anonymize data in the normal way, i assume you have custom host for your HEC source?
https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Anonymizedata
you can anonymize data based onhost,source or sourcetype
BUT
I would recommend option#1 , just apply a SEDCMD or sha256 function in whatever source code language your HEC event source is written in. You ask why?
Well,then the data remains safe in transit over the internet, assuming some one hacks into the message transfer protocol over internet all he/she would get is an encrypted string...

[lufermalgo]

Hi @Sukisen1981

Thank you very much for your answer.
Follow the instructions but it didn't work, this is my configuration:

[httpevent]
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SEDCMD-Anon = s/(\d+)\.(\d+)\.(\d+)\.(\d+)/\1.\2.xxx.xxx/g
category = Custom
disabled = false

This is HEC test:

curl -k "http://172.23.254.84:8088/services/collector/raw?sourcetype=httpevent&index=main" \
    -H "Authorization: Splunk cb985d67-c858-4951-a9d0-aed4bf614eda" \
        -d '127.0.0.1 - admin [26/Aug/2019:09:05:26.875 -0700] "GET /servicesNS/admin/launcher/data/ui/views?count=-1 HTTP/1.0" 200 126721 - - - 6ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.917 -0700] "GET /servicesNS/admin/launcher/data/ui/nav/default HTTP/1.0" 200 4367 - - - 6ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.941 -0700] "GET /services/apps/local?search=disabled%3Dfalse&count=-1 HTTP/1.0" 200 31930 - - - 4ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.954 -0700] "GET /services/apps/local?search=disabled%3Dfalse&count=-1 HTTP/1.0" 200 31930 - - - 3ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.968 -0700] "GET /servicesNS/admin/launcher/data/ui/views?digest=1&count=-1 HTTP/1.0" 200 58672 - - - 5ms'

What I am trying to do is hide the ip of the event, something like this: 127.0.xxx.xxx

My test environment is as follows:

![alt text][1]

Thank you

[mayurr98 · Accepted solution]

did you try this :

https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/Anonymizedata#Define_the_sed_script_in_props...

[lufermalgo]

Hi @mayurr98

Thank you very much for your answer.
Follow the instructions but it didn't work, this is my configuration:

[httpevent]
LINE_BREAKER = ([\r\n]+)
NO_BINARY_CHECK = true
SEDCMD-Anon = s/(\d+)\.(\d+)\.(\d+)\.(\d+)/\1.\2.xxx.xxx/g
category = Custom
disabled = false

This is HEC test:

curl -k "http://172.23.254.84:8088/services/collector/raw?sourcetype=httpevent&index=main" \
    -H "Authorization: Splunk cb985d67-c858-4951-a9d0-aed4bf614eda" \
        -d '127.0.0.1 - admin [26/Aug/2019:09:05:26.875 -0700] "GET /servicesNS/admin/launcher/data/ui/views?count=-1 HTTP/1.0" 200 126721 - - - 6ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.917 -0700] "GET /servicesNS/admin/launcher/data/ui/nav/default HTTP/1.0" 200 4367 - - - 6ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.941 -0700] "GET /services/apps/local?search=disabled%3Dfalse&count=-1 HTTP/1.0" 200 31930 - - - 4ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.954 -0700] "GET /services/apps/local?search=disabled%3Dfalse&count=-1 HTTP/1.0" 200 31930 - - - 3ms
        127.0.0.1 - admin [26/Aug/2019:09:05:26.968 -0700] "GET /servicesNS/admin/launcher/data/ui/views?digest=1&count=-1 HTTP/1.0" 200 58672 - - - 5ms'

What I am trying to do is hide the ip of the event, something like this: 127.0.xxx.xxx

My test environment is as follows:

Thank you

[mayurr98]

did you restart the server ? also this configuration will apply to the newer events only and not on the historical events.