strip sensitive data before indexing

zhatsispgx — Tue, 29 Sep 2020 18:52:17 GMT

Hi all,

We run several tools in our environment for network inspection and the logging it provides logs things like NTLM creds, HTTP Basic Auth etc. We'd like to strip this data off before indexing, or at least 'sanitize' it so we dont index usernames/passwords. Can anyone point me in the right direction to solve this?

Example data:

{
"timestamp":"2018-04-04T09:00:08.085563-0600",
"flow_id":151014950299099,
"in_iface":"asdfasdf",
"event_type":"alert",
"vlan":10,
"src_ip":"x.x.x.x",
"src_port":60130,
"dest_ip":"166.70.63.169",
"dest_port":443,
"proto":"TCP",
"tx_id":0,
"alert":{
"action":"allowed",
"gid":1,
"signature_id":2013928,
"rev":4,
"signature":"ET POLICY HTTP traffic on port 443 (PROPFIND)",
"category":"Potentially Bad Traffic",
"severity":2
},
"http":{
"hostname":"www.somesite.org",
"url":"\/things\/remote.php\/webdav\/",
"http_user_agent":"Mozilla\/5.0 (Linux) mirall\/2.3.3",
"http_content_type":"application\/xml",
"http_method":"PROPFIND",
"protocol":"HTTP\/1.1",
"status":207,
"length":382
},
"payload_printable":"PROPFIND \/owncloud\/remote.php\/webdav\/ HTTP\/1.1\r\n
Depth: 0\r\n
Authorization: Basic REDACTEDBASE64PASSWORDHERE==\r\n
User-Agent: Mozilla\/5.0 (Linux) mirall\/2.3.3\r\n
Accept: \/\r\n
Content-Type: text\/xml; charset=utf-8\r\nCookie: oc_sessionPassphrase=redacted \r\nContent-Length: 105\r\n
Connection: Keep-Alive\r\n
Accept-Encoding: gzip, deflate\r\nAccept-Language: en-US,*\r\n
Host: www.somesite.org\r\n\r\n\n\n \n \n <\/d:prop>\n<\/d:propfind>\n",
"stream":1
}

Re: strip sensitive data before indexing

richgalloway — Wed, 04 Apr 2018 16:29:07 GMT

See http://docs.splunk.com/Documentation/Splunk/7.0.3/Data/Anonymizedata

topic strip sensitive data before indexing in Getting Data In

strip sensitive data before indexing

Re: strip sensitive data before indexing