topic Re: Grabbing IPs from unstructure logs? in Getting Data In

Grabbing IPs from unstructure logs?

jharris1111 — Wed, 31 Jul 2013 18:08:50 GMT

I'm a splunk beginner, and have been able to do all kinds of interesting things with my logs that are structured as attribute=value pairs. The attribute names are discovered and I can graph the top 10 occurrences, etc.

What I'm having a harder time with is my logs that do not have attribute names built in. For example, my DHCP logs, I want to analyze IP address assignments, unusual MAC addresses, hostname to MAC or IP, etc. I've had some success with 'rex' on other logs, but not with these. Is rex the best/easiest way? Other suggestions for learning the fields I want?

Jul 31 10:53:52 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.164 00:26:08:68:6c:8a

Jul 31 10:53:52 dnsmasq-dhcp[24852]: DHCPREQUEST(br0) 10.31.11.164 00:26:08:68:6c:8a

Jul 31 10:24:32 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.249 00:18:dd:33:0e:f4 HDHR-2140DF1C

Jul 31 08:30:55 dnsmasq-dhcp[24852]: DHCPACK(br0) 10.31.11.107 00:23:6c:bb:f4:15 wifi-extreme

Re: Grabbing IPs from unstructure logs?

alacercogitatus — Wed, 31 Jul 2013 18:26:01 GMT

Try this Regex in your rex command:

your_search | rex field=_raw "\w+\s\d+\s\d+:\d+:\d+\s(?<process_name>[^\[]*)\[(?<pid>\d+)\]:\s(?<action>[^$]*)\((?<interface>[^$]*)\)\s(?<src_ip>[^\s]*)\s(?<src_mac>[^\s]*)\s(?<src_host>[^\s]*)"

You can also do it automatically in props/transforms.

props.conf [sourcetypeForDHCP] REPORT-0auto_fields = dhcp-field-ext

transforms.conf [dhcp-field-ext] REGEX = \w+\s\d+\s\d+:\d+:\d+\s([^\[]*)\[(\d+)\]:\s([^$]*)\(([^$]*)\)\s([^\s]*)\s([^\s]*)\s([^\s]*) FORMAT = process_name::$1 pid::$2 action::$3 interface::$4 src_ip::$5 src_mac::$6 src_host::$7 MV_ADD = true

Re: Grabbing IPs from unstructure logs?

jharris1111 — Thu, 01 Aug 2013 22:04:50 GMT

Thanks, alacercogitatus.

This worked great and I was able to re-use this solution for some similar analysis.

Re: Grabbing IPs from unstructure logs?

jspears — Sun, 08 Jan 2017 13:52:08 GMT

This worked for me after changing the beginning of the rex to "\w+\s+" , using this against dnmasq-dhcp syslog from Tomato firmware.