Log Format

[wbkendall]

After my other question, I installed Kiwi Syslog server on my Windows box and have it set up to receive syslog messages from the network and log to a file. I pointed Splunk to the logfile location and set it to feed into this application.

No matter what logfile format I choose in Kiwi, it looks like some of the fields don't line up. For instance, take this event from Kiwi:

02-05-2014 09:34:10 User.Alert 192.168.1.1 Feb 5 09:32:40 kernel: DROP <4>DROPIN=eth0 OUT= MAC=e0:cb:4e:c4:dd:24:00:01:5c:64:4e:46:08:00 <1>SRC=4.79.142.206 DST=96.29.46.xxx <1>LEN=44 TOS=0x00 PREC=0x00 TTL=225 ID=61440 PROTO=TCP <1>SPT=61690 DPT=23 SEQ=56734009 ACK=0 WINDOW=8192 RES=0x00 SYN URGP=0 OPT (020405B4)

When this event shows up in the app, it has an "unknown" value for the source IP address.

Logs are coming from a router running DD-WRT. My guess is that either the Kiwi service is writing syslog files in a format that doesn't match native iptables, or the router is sending logs in a format that doesn't match native iptables.

Either way I'm out of ideas.

Thanks!

[sbrant_splunk · Accepted solution]

Are you seeing a field named "SRC"? Or a field named "<1>SRC"?

It looks like you've got some extraneous characters in the log. You can adjust the regular expression that pulls out source_ip, try this:

(?i)\s*SRC=(?P<source_ip>[^\s]+)

That should take care of the source_ip but it looks like the app is dependent on auto-extraction via the key=value pairs that are in the log. To fix those, you'll need to add some manual extractions to props.conf.

BTW, don't change the default props.conf, instead copy it to the local directory in the iptables app and make the changes there.

[wbkendall]

Yes, looks like logs aren't in standard format. Here's a DROP event:
Feb 6 17:16:48 192.168.1.1 Feb 6 17:15:15 kernel: DROP <4>DROPIN=eth0 OUT= MAC=e0:cb:4e:c4:dd:24:00:01:5c:64:4e:46:08:00 <1>SRC=198.20.69.74 DST=96.29.x.x <1>LEN=40 TOS=0x00 PREC=0x00 TTL=117 ID=53857 PROTO=TCP <1>SPT=19139 DPT=80 SEQ=1395526512 ACK=0 WINDOW=31849 RES=0x00 SYN URGP=0

I'm trying to find out how to set up extractions in props.conf but this is my second day of Splunking and it's slow going. 🙂