Security

How to ingest Untangle logs (NG Firewall) into Splunk?

Engager

Greetings,

For a long time, I have wanted to ingest untangle logs (Firewall, IDS/IPS, OpenVpn, and Web Filtering) into Splunk to write security rules, etc. I am surprised this wasn't done before however I completed this and it was worth struggle.

1 Solution

Engager
  1. Getting Splunk and Untangle ready;

Pre-reqs (Ubuntu 16.04 - Splunk box) - JRE version 1.8 [I already had default jre]

sudo apt-get install default-jre [I would if I were you because these are the exact steps i followed]

http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html

tar -xvzf jre-8u172-linux-x64.tar.gz [change version to fit your needs]

mv jre1.8.0_172/ /usr/lib/jvm/

  1. Installing the splunk app.

Login to splunk base and download the DB connect app. Once you have installed it on the general tab under settings put the full path to your jave jre (/usr/lib/jvm/jre1.8.0_172)

Then download the driver required by the splunk app; https://jdbc.postgresql.org/download/postgresql-42.2.2.jar

After you have downloaded the correct driver for your database, copy the .JAR driver file to the

$SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers

Once you do this restart splunk

Configuring untangle;

  1. /etc/init.d/untangle-vm stop

  2. /etc/init.d/postgresql stop

Use fav text editor to edit

  1. /etc/postgresql/9.4/main/pg_hba.conf

Find this line;

host    all             all             127.0.0.1/0               trust

CHANGE It to

host    all             all             0.0.0.0/0               trust

This allows all traffic to it.

For this next part, navigate to

/etc/postgresql/9.6/main/postgresql.conf

UNCOMMENT the listen_address line and add * in the ().

Configure postgres;

psql -U postgres -d uvm
CREATE USER $usernamehere WITH ENCRYPTED PASSWORD 'passwordfortheuser';
GRANT CONNECT ON DATABASE uvm TO $usernamehere;
GRANT USAGE ON SCHEMA reports TO $usernamehere;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA reports TO $usernamehere;
GRANT SELECT ON ALL TABLES IN SCHEMA reports to $usernamehere;
ALTER DEFAULT PRIVILEGES IN SCHEMA reports GRANT SELECT ON TABLES TO $usernamehere;

IF YOU HAVE UNTANGLE FIREWALL RULES ENSURE THAT YOU ALLOW YOUR HOST ACCESS TO IT!
Download this tool to test https://www.pgadmin.org/download
Follow instructions to connect.

Tables to pull data from (this is what I did, you can tailor this to your environment);

[LIST]
[]openvpn_stats
[
]sessions
[]http_query events
[
]httpevents
[*]intrusion
prevention_events
[/LIST]

For configuring Splunk DB_Connect App and any questions please watch
https://youtu.be/oPB2Lpd9ZAs

Once you have the Splunk DB_Connect App setup:

You can use these pre-made queries to help get you up and running, the are splunk CIM compliant (except for OpenVPN).

OPENVPN;

SELECT stats.starttime AS sessionstarttime,
stats.end
time AS sessionclosetime,
stats.remoteaddress AS src,
stats.pool
address AS internalip,
stats.client
name AS user,
stats.rxbytes AS bytesin,
stats.txbytes AS bytesout,
event."type" AS action
FROM "uvm"."reports"."openvpnstats" AS stats
INNER JOIN "uvm"."reports"."openvpn
events" AS event
ON (stats.remoteaddress = event.remoteaddress)
AND (stats.clientname = event.clientname)
AND (stats.timestamp >= event.timestamp - INTERVAL '1' SECOND)

Firewall and SSL;

SELECT timestamp AS starttime,
endtime,
bypassed,
session
id,
hostname,
localaddr AS srcip,
cclientport AS srcport,
remote
addr AS destip,
c
serverport AS destport,
servercountry,
server
latitude,
serverlongitude,
c2p
bytes AS bytesout,
s2p
bytes AS bytesin,
firewall
blocked AS action,
sslinspectorruleid AS sslrule,
ssl
inspectorstatus AS sslaction,
sslinspectordetail AS ssl_url
FROM "uvm"."reports"."sessions"

IDS;

SELECT timestamp AS starttime,
sigid,
source
addr AS src,
destaddr AS dest,
dest
port AS destport,
blocked AS action,
category || ':' || classtype AS category,
msg AS signature
FROM "uvm"."reports"."intrusion
prevention_events"

Web_Filtering

SELECT httpevents.timestamp,
httpevents.cclientaddr AS src,
http
events.sserveraddr AS dest,
httpevents.host AS site,
http
events.host || httpevents.uri AS url,
http
events.domain AS destdomain,
http
events.hostname AS host,
httpevents.method AS httpmethod,
httpevents.s2ccontenttype AS httpcontenttype,
http
events.referer AS httpreferrer,
http
events.webfiltercategory AS category,
httpqueryevents.uri AS uriquery,
http
queryevents.term AS searchterms
FROM "uvm"."reports"."httpevents" AS httpevents
INNER JOIN "uvm"."reports"."httpqueryevents" AS httpqueryevents
ON (httpevents.requestid = httpqueryevents.requestid)
AND (http
events.sessionid = httpqueryevents.sessionid)

View solution in original post

Engager
  1. Getting Splunk and Untangle ready;

Pre-reqs (Ubuntu 16.04 - Splunk box) - JRE version 1.8 [I already had default jre]

sudo apt-get install default-jre [I would if I were you because these are the exact steps i followed]

http://www.oracle.com/technetwork/java/javase/downloads/jre8-downloads-2133155.html

tar -xvzf jre-8u172-linux-x64.tar.gz [change version to fit your needs]

mv jre1.8.0_172/ /usr/lib/jvm/

  1. Installing the splunk app.

Login to splunk base and download the DB connect app. Once you have installed it on the general tab under settings put the full path to your jave jre (/usr/lib/jvm/jre1.8.0_172)

Then download the driver required by the splunk app; https://jdbc.postgresql.org/download/postgresql-42.2.2.jar

After you have downloaded the correct driver for your database, copy the .JAR driver file to the

$SPLUNK_HOME/etc/apps/splunk_app_db_connect/drivers

Once you do this restart splunk

Configuring untangle;

  1. /etc/init.d/untangle-vm stop

  2. /etc/init.d/postgresql stop

Use fav text editor to edit

  1. /etc/postgresql/9.4/main/pg_hba.conf

Find this line;

host    all             all             127.0.0.1/0               trust

CHANGE It to

host    all             all             0.0.0.0/0               trust

This allows all traffic to it.

For this next part, navigate to

/etc/postgresql/9.6/main/postgresql.conf

UNCOMMENT the listen_address line and add * in the ().

Configure postgres;

psql -U postgres -d uvm
CREATE USER $usernamehere WITH ENCRYPTED PASSWORD 'passwordfortheuser';
GRANT CONNECT ON DATABASE uvm TO $usernamehere;
GRANT USAGE ON SCHEMA reports TO $usernamehere;
GRANT SELECT ON ALL SEQUENCES IN SCHEMA reports TO $usernamehere;
GRANT SELECT ON ALL TABLES IN SCHEMA reports to $usernamehere;
ALTER DEFAULT PRIVILEGES IN SCHEMA reports GRANT SELECT ON TABLES TO $usernamehere;

IF YOU HAVE UNTANGLE FIREWALL RULES ENSURE THAT YOU ALLOW YOUR HOST ACCESS TO IT!
Download this tool to test https://www.pgadmin.org/download
Follow instructions to connect.

Tables to pull data from (this is what I did, you can tailor this to your environment);

[LIST]
[]openvpn_stats
[
]sessions
[]http_query events
[
]httpevents
[*]intrusion
prevention_events
[/LIST]

For configuring Splunk DB_Connect App and any questions please watch
https://youtu.be/oPB2Lpd9ZAs

Once you have the Splunk DB_Connect App setup:

You can use these pre-made queries to help get you up and running, the are splunk CIM compliant (except for OpenVPN).

OPENVPN;

SELECT stats.starttime AS sessionstarttime,
stats.end
time AS sessionclosetime,
stats.remoteaddress AS src,
stats.pool
address AS internalip,
stats.client
name AS user,
stats.rxbytes AS bytesin,
stats.txbytes AS bytesout,
event."type" AS action
FROM "uvm"."reports"."openvpnstats" AS stats
INNER JOIN "uvm"."reports"."openvpn
events" AS event
ON (stats.remoteaddress = event.remoteaddress)
AND (stats.clientname = event.clientname)
AND (stats.timestamp >= event.timestamp - INTERVAL '1' SECOND)

Firewall and SSL;

SELECT timestamp AS starttime,
endtime,
bypassed,
session
id,
hostname,
localaddr AS srcip,
cclientport AS srcport,
remote
addr AS destip,
c
serverport AS destport,
servercountry,
server
latitude,
serverlongitude,
c2p
bytes AS bytesout,
s2p
bytes AS bytesin,
firewall
blocked AS action,
sslinspectorruleid AS sslrule,
ssl
inspectorstatus AS sslaction,
sslinspectordetail AS ssl_url
FROM "uvm"."reports"."sessions"

IDS;

SELECT timestamp AS starttime,
sigid,
source
addr AS src,
destaddr AS dest,
dest
port AS destport,
blocked AS action,
category || ':' || classtype AS category,
msg AS signature
FROM "uvm"."reports"."intrusion
prevention_events"

Web_Filtering

SELECT httpevents.timestamp,
httpevents.cclientaddr AS src,
http
events.sserveraddr AS dest,
httpevents.host AS site,
http
events.host || httpevents.uri AS url,
http
events.domain AS destdomain,
http
events.hostname AS host,
httpevents.method AS httpmethod,
httpevents.s2ccontenttype AS httpcontenttype,
http
events.referer AS httpreferrer,
http
events.webfiltercategory AS category,
httpqueryevents.uri AS uriquery,
http
queryevents.term AS searchterms
FROM "uvm"."reports"."httpevents" AS httpevents
INNER JOIN "uvm"."reports"."httpqueryevents" AS httpqueryevents
ON (httpevents.requestid = httpqueryevents.requestid)
AND (http
events.sessionid = httpqueryevents.sessionid)

View solution in original post