In my log data I get lines that look like this:
dst=10.0.59.59:80:X1
dst=255.255.255.255:67:X0
dst=10.0.59.59:9060:X1
dst=0.0.0.0:0:X0
dst=224.0.0.5:1
The first value is an IP address. The next two values should be port number and interface. I did some field extractions and I can get it to extract all three fields if they are present but when it has only 2 fields it throws away the data. Could you help me write a field extraction regex to get the 2 field and 3 field variants.
the field names should be dst_ip, dst_port, dst_interface
Well, since there might not always be three fields to extract from the same piece of data, I'd do it in two EXTRACTs in props.conf
[your sourcetype]
EXTRACT-dst_ip_port = \s+dst=(?<dst_ip>[^:]+):(?<dst_port>\d+)
EXTRACT-dst_if = \s+dst=[^:]+:\d+:(?<dst_interface>\S+)
/K
that's why I put in \S+
, i.e. 'one or more non-whitespace characters' for the interface extraction. If this indeed captures more than intended you should post a couple of full events, or explain more clearly how the fields are delimited in your events.
I would use a transform for this one. I not sure how you event is seperated, this is for space delim event. Using Transforms you can create additional extraction from an already extracted value.
example Event : 2012-04-23 13:24:25 SUCCESS 10.0.59.59:9060:2561X0 10.214.1.79:9060:X1
First trasnfrom does the following key value pairs from _raw:
The second transform does the following by using dst as the source_key to prefrom extraction.
#transforms.conf
[some_event]
DELIMS = " "
FIELDS = date,time,status,scr,dst
[dstextract]
SOURCE_KEY = dst
DELIM = ":"
FIELDS = ip,port,interface
#props.conf
[Mysource]
MAX_TIMESTAMP_LOOKAHEAD=40
NO_BINARY_CHECK=1
SHOULD_LINEMERGE=false
TZ=US/Pacific
REPORT-Mysource=some_event,dstextract
Hope this help or gives you some ideas. Dont forget to accept and vote up answers that up.
Cheers,
I preform all configurations through direct edit of the conf files. The GUI only provides limited functionality for advanced configurations direct editing is required.
this looks good. I see that you posted to do it in flat text. Is there any way to do this in the GUI so I can test the functionality?
for searching for dst_ip and dst_port this seems to work
(?i) dst=(?P
No idea if that's efficient
this one is a little better
(?i) dst=[^:]+:\d+:(?P
I found out we have other things then just X and possibly 2 digits
oh this seems to work
(?i) dst=[^:]+:\d+:(?P
Well, since there might not always be three fields to extract from the same piece of data, I'd do it in two EXTRACTs in props.conf
[your sourcetype]
EXTRACT-dst_ip_port = \s+dst=(?<dst_ip>[^:]+):(?<dst_port>\d+)
EXTRACT-dst_if = \s+dst=[^:]+:\d+:(?<dst_interface>\S+)
/K
yes you must do a splunk restart
when I change the props.conf do I need to restart splunk?
The regexes I listed above are meant to go into props.conf.
Also, when posting, use the backtick (`) around code examples. otherwise a lot of stuff will be filtered out.
when I remove the "dst=" and add colons to be beginning and end so it looks like this:
:10.0.59.59:80:X1:
The regex generated looks like:
(?:[^ \n]* ){3}(?P
When I put your regex into the Interactive field extractor I get nothing. Does the \s+dst need a different beginning?