How can I use Enterprise Security blocklists with Squid?

[jcoates_splunk]

I've got some dynamically gathered blocklist information in Splunk App for Enterprise Security... and I want to put those lists into my Squid configuration.

[jcoates_splunk · Accepted solution]

There are lots of tools to enable blocklist gathering in Squid, but since we've already gathered the data, why not just use the stuff that's already here? I use this script on my home server. That destination path is a little funny because I use SquidMan to manage my configuration.

squid.conf change:

acl blocklistIP src "Users/coates/Library/Preferences/squid_blocklist.txt"
http_access deny blocklistIP

shell script (runs via cron):

#!/bin/sh

if [ ! -z "$SPLUNK_HOME" ]
    then if [ -e "/Applications/splunk/" ] 
        then export SPLUNK_HOME=/Applications/splunk
    elif [ -e "/opt/splunk" ]
        then export SPLUNK_HOME=/Applications/splunk
    else
        echo "Please set \$SPLUNK_HOME"
    fi
fi

SRC=$SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/lookups
DEST=$HOME/Library/Preferences/squid_blocklist.txt

rm -rf $DEST

# Ranges
cat $SRC/ip_piratebaylist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/ip_rapidsharelist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/sans_blocklist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST

# Solo addresses
cat $SRC/ip_proxylist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_spywarelist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_torlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_webattackerlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST

sort $DEST | uniq > $DEST.tmp

cat $DEST.tmp | grep - | awk 'BEGIN {FS="."} {print $1"."$2"."$3}' > /tmp/badnets.txt
while read line; do grep -v $line $DEST.tmp |grep -v -; done < /tmp/badnets.txt > $DEST.tmp2

sort $DEST.tmp2 | uniq > $DEST

rm $DEST.tmp
rm $DEST.tmp2
rm /tmp/badnets.txt