Getting Data In

Grabbing IPs from unstructure logs?

jharris1111
Engager

I'm a splunk beginner, and have been able to do all kinds of interesting things with my logs that are structured as attribute=value pairs. The attribute names are discovered and I can graph the top 10 occurrences, etc.

What I'm having a harder time with is my logs that do not have attribute names built in. For example, my DHCP logs, I want to analyze IP address assignments, unusual MAC addresses, hostname to MAC or IP, etc. I've had some success with 'rex' on other logs, but not with these. Is rex the best/easiest way? Other suggestions for learning the fields I want?

Jul 31 10:53:52 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.164 00:26:08:68:6c:8a

Jul 31 10:53:52 dnsmasq-dhcp[24852]: DHCPREQUEST(br0) 10.31.11.164 00:26:08:68:6c:8a

Jul 31 10:24:32 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.249 00:18:dd:33:0e:f4 HDHR-2140DF1C

Jul 31 08:30:55 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.107 00:23:6c:bb:f4:15 wifi-extreme

0 Karma

alacercogitatus
SplunkTrust
SplunkTrust

Try this Regex in your rex command:

your_search | rex field=_raw "\w+\s\d+\s\d+:\d+:\d+\s(?<process_name>[^\[]*)\[(?<pid>\d+)\]:\s(?<action>[^\(]*)\((?<interface>[^\)]*)\)\s(?<src_ip>[^\s]*)\s(?<src_mac>[^\s]*)\s(?<src_host>[^\s]*)"

You can also do it automatically in props/transforms.

props.conf
[sourcetypeForDHCP]
REPORT-0auto_fields = dhcp-field-ext

transforms.conf
[dhcp-field-ext]
REGEX = \w+\s\d+\s\d+:\d+:\d+\s([^\[]*)\[(\d+)\]:\s([^\(]*)\(([^\)]*)\)\s([^\s]*)\s([^\s]*)\s([^\s]*)
FORMAT = process_name::$1 pid::$2 action::$3 interface::$4 src_ip::$5 src_mac::$6 src_host::$7
MV_ADD = true

jharris1111
Engager

Thanks, alacercogitatus.

This worked great and I was able to re-use this solution for some similar analysis.

0 Karma

jspears
Communicator

This worked for me after changing the beginning of the rex to "\w+\s+" , using this against dnmasq-dhcp syslog from Tomato firmware.

0 Karma
.conf21 Now Fully Virtual!
Register for FREE Today!

We've made .conf21 totally virtual and totally FREE! Our completely online experience will run from 10/19 through 10/20 with some additional events, too!