All Apps and Add-ons

Help extracting fields (IPs and Ports) from a specific syslog message

healthtrans
Explorer

Can anyone assist with extracting the IP addresses and ports from this syslog message? I tried the 'extract fields' tool but was not successful.

Jul 21 14:09:23 192.168.1.1 HOSTNAME: NetScreen device_id=HOSTNAME [Root]system-alert-00016: Port scan! From 1.1.1.1:80 to 2.2.2.2:7136, proto TCP (zone Untrust int ethernet3). Occurred 1 times. (2011-07-21 09:09:17)

Thanks.

0 Karma

fk319
Builder

use props.conf to call transforms.conf


you can then build your regex

# props.conf
[source::fromSomewhere]
TRANSFORMS-getIP = from-to-ips

# transforms.conf
[from-to-ips]
    # From 1.1.1.1:80 to 2.2.2.2:7136, proto 
    REGEX = From ([0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\:[0-9]{1-5}) to ([0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\.[0-9]{1-3}\:[0-9]{1-5}), proto
    FORMAT = FromIP::$1 ToIP::$2

depending on how strict you want your ip:port matching to be, you can tighten your regex.

0 Karma

Drainy
Champion

You could build a custom transform.
http://www.splunk.com/base/Documentation/4.2.2/Data/Advancedsourcetypeoverrides

I've been playing around with this alot lately.
For example...

16:31:55.879529 00:16:0a:0b:92:fb >
ff:ff:ff:ff:ff:ff, ethertype IPv4
(0x0800), length 150:
192.168.3.2.42090 > 192.168.3.255.111: UDP, length 108

Is formatted with;

[tcpdump_basic] REGEX = ([^ ]+)([
])([^ ]+) ([>]) ([^,]+)([^ ]) ([^ ]+)
([^ ]+) ([^ ]+) ([^ ]+[^:]+) ([^ ]+)
([1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2})([.]+)([^>]+)
([>])
([1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2}.[1-2]{0,1}[0-9]{1,2})([.]+)([^:]+)([^
]+) ([^ ]+)([,]+)

FORMAT = timestamp::$1 src_mac::$3
dest_mac::$5 net_layer::$8
source_host::$12 source_port::$14
destin_host::$16 destin_port::$18
protocol::$20

The above goes in transforms.conf and then I just pop the following bit in props.conf

REPORT-tcpdump_basic = tcpdump_basic

You need to do a little bit more in props to define a sourcetype but you get the general idea. Don't let the regex scare you either. Copy and paste the regex and my example text to http://gskinner.com/RegExr/ and then hover over the highlighted output. It breaks down what each group relates to.
Now in my search window it correctly picks out all the right fields with the names I've defined.

Maybe slightly easier - there is a new version of the field extractor on apps that is apparently alot better than the baked in one if you don't already have it

EDIT: Link to the new field extractor

0 Karma
Got questions? Get answers!

Join the Splunk Community Slack to learn, troubleshoot, and make connections with fellow Splunk practitioners in real time!

Meet up IRL or virtually!

Join Splunk User Groups to connect and learn in-person by region or remotely by topic or industry.

Get Updates on the Splunk Community!

Unlocking Unified Insights: New Gigamon Federated Search App for Splunk

In today’s data-heavy environment, organizations are caught in a data distribution dilemma. As data volumes ...

GA: New Data Management App in Splunk Platform

Streamlining Data Management: Introducing a unified experience in Splunk Managing data at scale shouldn’t feel ...

Announcing Modern Navigation: A New Era of Splunk User Experience

We are excited to introduce the Modern Navigation feature in the Splunk Platform, available to both cloud and ...