Security

How can I use Enterprise Security blocklists with Squid?

jcoates_splunk
Splunk Employee
Splunk Employee

I've got some dynamically gathered blocklist information in Splunk App for Enterprise Security... and I want to put those lists into my Squid configuration.

0 Karma
1 Solution

jcoates_splunk
Splunk Employee
Splunk Employee

There are lots of tools to enable blocklist gathering in Squid, but since we've already gathered the data, why not just use the stuff that's already here? I use this script on my home server. That destination path is a little funny because I use SquidMan to manage my configuration.

squid.conf change:

acl blocklistIP src "Users/coates/Library/Preferences/squid_blocklist.txt"
http_access deny blocklistIP

shell script (runs via cron):

#!/bin/sh

if [ ! -z "$SPLUNK_HOME" ]
    then if [ -e "/Applications/splunk/" ] 
        then export SPLUNK_HOME=/Applications/splunk
    elif [ -e "/opt/splunk" ]
        then export SPLUNK_HOME=/Applications/splunk
    else
        echo "Please set \$SPLUNK_HOME"
    fi
fi

SRC=$SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/lookups
DEST=$HOME/Library/Preferences/squid_blocklist.txt

rm -rf $DEST

# Ranges
cat $SRC/ip_piratebaylist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/ip_rapidsharelist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/sans_blocklist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST

# Solo addresses
cat $SRC/ip_proxylist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_spywarelist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_torlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_webattackerlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST

sort $DEST | uniq > $DEST.tmp

cat $DEST.tmp | grep - | awk 'BEGIN {FS="."} {print $1"."$2"."$3}' > /tmp/badnets.txt
while read line; do grep -v $line $DEST.tmp |grep -v -; done < /tmp/badnets.txt > $DEST.tmp2

sort $DEST.tmp2 | uniq > $DEST

rm $DEST.tmp
rm $DEST.tmp2
rm /tmp/badnets.txt

View solution in original post

jcoates_splunk
Splunk Employee
Splunk Employee

There are lots of tools to enable blocklist gathering in Squid, but since we've already gathered the data, why not just use the stuff that's already here? I use this script on my home server. That destination path is a little funny because I use SquidMan to manage my configuration.

squid.conf change:

acl blocklistIP src "Users/coates/Library/Preferences/squid_blocklist.txt"
http_access deny blocklistIP

shell script (runs via cron):

#!/bin/sh

if [ ! -z "$SPLUNK_HOME" ]
    then if [ -e "/Applications/splunk/" ] 
        then export SPLUNK_HOME=/Applications/splunk
    elif [ -e "/opt/splunk" ]
        then export SPLUNK_HOME=/Applications/splunk
    else
        echo "Please set \$SPLUNK_HOME"
    fi
fi

SRC=$SPLUNK_HOME/etc/apps/SA-ThreatIntelligence/lookups
DEST=$HOME/Library/Preferences/squid_blocklist.txt

rm -rf $DEST

# Ranges
cat $SRC/ip_piratebaylist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/ip_rapidsharelist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST
cat $SRC/sans_blocklist.csv | awk 'BEGIN {FS=","} {print $1"/32"}' | tr -d '"' | grep -v [:numeric] >> $DEST

# Solo addresses
cat $SRC/ip_proxylist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_spywarelist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_torlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST
cat $SRC/ip_webattackerlist.csv | awk 'BEGIN {FS=","} {print $1}' | awk 'BEGIN {FS="-"} {print $1"/32"}' | tr -d '"'  | grep -v [:numeric] >> $DEST

sort $DEST | uniq > $DEST.tmp

cat $DEST.tmp | grep - | awk 'BEGIN {FS="."} {print $1"."$2"."$3}' > /tmp/badnets.txt
while read line; do grep -v $line $DEST.tmp |grep -v -; done < /tmp/badnets.txt > $DEST.tmp2

sort $DEST.tmp2 | uniq > $DEST

rm $DEST.tmp
rm $DEST.tmp2
rm /tmp/badnets.txt
Get Updates on the Splunk Community!

Splunk Forwarders and Forced Time Based Load Balancing

Splunk customers use universal forwarders to collect and send data to Splunk. A universal forwarder can send ...

NEW! Log Views in Splunk Observability Dashboards Gives Context From a Single Page

Today, Splunk Observability releases log views, a new feature for users to add their logs data from Splunk Log ...

Last Chance to Submit Your Paper For BSides Splunk - Deadline is August 12th!

Hello everyone! Don't wait to submit - The deadline is August 12th! We have truly missed the community so ...