Splunk Search

scrub IP only

rstanonik
Engager

I'm tasked to provide apache logs to a third party for their analysis, but the IPs must be replaced to hide the browsers' identity.
Sounds like a simple splunk job: select, piped through scrub, then exported.
But the scrub function scrubs more than just the IPs, it also scrubs the URLs.
I'm hunting for alternatives that will only scrub IPs.
Alternatively, I'm considering hacking scrub to only affect IPs.
Any thoughts?
Thanks.

Tags (3)

d3
Explorer

You can use a rename within your search to temporarily hide what you don't want scrubbed, then rename it again after the scrub but before the results are presented. The example below is something we've come up with to scrub a firewall IPS log. The search looks for the device (FG is a FortiGate) and message type (ips). The "ref" is a reference to a real URL from the vendor website. We rename the ref to _ref which gets ignored by scrub, then rename _ref to ref and build the report table.

device_id="FG*" type="ips" | stats count by msg devname ref | rename ref as _ref | scrub | sort 10 -count | rename _ref as ref| table msg devname ref

0 Karma

MarioM
Motivator

do you do it on the ip field? | scrub ipfieldname

0 Karma

chris
Motivator

One way of getting a result although it is not very elegant:

  1. Export the Events
  2. Save a list of the ip adresses ( append '|dedup ip-field| table ip-field' to your search)
  3. Run $SPLUNK_HOME/bin/splunk anonymize file -source /tmp/events.txt -private-terms /opt/splunk/etc/anonymizer/ip-list.txt -public-terms /tmp/events.txt

The public-terms wont get replaced (which is everything since the file with the events is used) the private-terms do get replaced and they seem to have a higher priority

I got the Idea from here:
http://docs.splunk.com/Documentation/Splunk/latest/Troubleshooting/AnonymizedatasamplestosendtoSuppo...

Not very nice, not very efficient I know, but it worked ...

dwaddle
SplunkTrust
SplunkTrust

Better choice may be to use rex in sed mode. This is a rough stab at a sed-expression to do some (not optimal) scrubbing.

... | rex mode=sed "s/([0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3})\.([0-9]{1,3})/xxx.yyy.zzz.\2/g"
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...