Regex for extracting ip port and interface

[jalfrey]

In my log data I get lines that look like this:
dst=10.0.59.59:80:X1
dst=255.255.255.255:67:X0
dst=10.0.59.59:9060:X1
dst=0.0.0.0:0:X0
dst=224.0.0.5:1

The first value is an IP address. The next two values should be port number and interface. I did some field extractions and I can get it to extract all three fields if they are present but when it has only 2 fields it throws away the data. Could you help me write a field extraction regex to get the 2 field and 3 field variants.

the field names should be dst_ip, dst_port, dst_interface

[kristian_kolb]

that's why I put in \S+, i.e. 'one or more non-whitespace characters' for the interface extraction. If this indeed captures more than intended you should post a couple of full events, or explain more clearly how the fields are delimited in your events.

[bmacias84]

I would use a transform for this one. I not sure how you event is seperated, this is for space delim event. Using Transforms you can create additional extraction from an already extracted value.

example Event : 2012-04-23 13:24:25 SUCCESS 10.0.59.59:9060:2561X0 10.214.1.79:9060:X1

First trasnfrom does the following key value pairs from _raw:

• date=2012-04-23
• time=13:24:25
• status=SUCCESS
• scr=10.0.59.59:9060:2561X0
• dst=10.214.1.79:9060:X1

The second transform does the following by using dst as the source_key to prefrom extraction.

• ip=10.214.1.79
• port=9060
• interface=X1

  
  #transforms.conf
  [some_event]
  DELIMS = " "
  FIELDS = date,time,status,scr,dst
  [dstextract]
  SOURCE_KEY = dst
  DELIM = ":"
  FIELDS = ip,port,interface
  

  
  #props.conf
  [Mysource]
  MAX_TIMESTAMP_LOOKAHEAD=40
  NO_BINARY_CHECK=1
  SHOULD_LINEMERGE=false
  TZ=US/Pacific
  REPORT-Mysource=some_event,dstextract
  

Hope this help or gives you some ideas. Dont forget to accept and vote up answers that up.

Cheers,

[bmacias84]

I preform all configurations through direct edit of the conf files. The GUI only provides limited functionality for advanced configurations direct editing is required.

[jalfrey]

this looks good. I see that you posted to do it in flat text. Is there any way to do this in the GUI so I can test the functionality?

[jalfrey]

for searching for dst_ip and dst_port this seems to work
(?i) dst=(?P\d+.\d+.\d+.\d+):(?P[0-9]+)
No idea if that's efficient

[jalfrey]

this one is a little better
(?i) dst=[^:]+:\d+:(?P\w[0-9]+)
I found out we have other things then just X and possibly 2 digits

[jalfrey]

oh this seems to work
(?i) dst=[^:]+:\d+:(?PX[0-9])

[kristian_kolb · Accepted solution]

Well, since there might not always be three fields to extract from the same piece of data, I'd do it in two EXTRACTs in props.conf

[your sourcetype]
EXTRACT-dst_ip_port = \s+dst=(?<dst_ip>[^:]+):(?<dst_port>\d+)
EXTRACT-dst_if = \s+dst=[^:]+:\d+:(?<dst_interface>\S+)

/K

[jalfrey]

yes you must do a splunk restart

[jalfrey]

when I change the props.conf do I need to restart splunk?

[kristian_kolb]

The regexes I listed above are meant to go into props.conf.

Also, when posting, use the backtick (`) around code examples. otherwise a lot of stuff will be filtered out.

[jalfrey]

when I remove the "dst=" and add colons to be beginning and end so it looks like this:
:10.0.59.59:80:X1:

The regex generated looks like:
(?:[^ \n]* ){3}(?P[^ ]+)(?:[^.\n].){5}\d+(?P\d+)(?:[^:\n]:){2}(?P[^ ]+)

When I put your regex into the Interactive field extractor I get nothing. Does the \s+dst need a different beginning?