topic DELIM does not work in Splunk Search

DELIM does not work

wsnyder2 — Mon, 28 Sep 2020 14:23:46 GMT

We can not get field extraction to work with IIS log files. Any suggestions?

transforms.conf
[iisw3cfields]
DELIMS = " "
FIELDS = date,time,s-computername,s-ip,cs-method,cs-uri-stem,cs-uri-query,s-port,cs-username,c-ip,cs-version,cs(User-Agent),cs(Cookie),cs(Referer),cs-host,sc-status,sc-bytes,cs-bytes,time-taken

props.conf (relevant part)
...
[iis_ex]
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 19
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TZ = GMT
LINE_BREAKER = ([\r\n]+)(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})
SHOULD_LINEMERGE = False
TRUNCATE = 10000
KV_MODE = None
pulldown_type = true
REPORT-iisw3cfields = iisw3cfields

Here are a few lines from our log file ...

Software: Microsoft Internet Information Services 6.0

Version: 1.0

Date: 2013-07-07 00:00:00

Fields: date time s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-bytes cs-bytes time-taken

2013-07-06 23:59:59 TLOWPN29 192.168.122.82 GET /secure/EmployeeHome.aspx - 443 - 207.190.231.139 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.2;+WOW64;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) training=true;+SessionGUID=ee81cbbf-9da8-4532-92ae-24a4e8ad6147;+StyleSheetTheme=pxxxx;+SSODeferral=true hxxx://xxxxx.xxxx.com/secure/login.aspx?alias=0414T735 xxxxxxx.xxxxx.com 200 68499 649 1109

2013-07-07 00:00:00 TLOWPN29 192.168.122.82 GET /secure/Images/exception.gif - 443 - 38.102.232.83 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) StyleSheetTheme=pxxxx;+SessionGUID=00000000-0000-0000-0000-000000000000;+training=true hxxps://xxxxxxx.xxxx.com/secure/login.aspx tixxxxxx.xxxxx.com 200 1338 597 15

Re: DELIM does not work

linu1988 — Fri, 19 Jul 2013 21:05:27 GMT

i think it's overcooked. Let it flow into splunk. Splunk should be able to get the events divided whereever it finds time at the start. After that, do the field extraction on the UI, which is easier, where you can actually see the matching. If the time is not correct, set the Props.conf for time recognition. Thanks

Re: DELIM does not work

bmacias84 — Fri, 19 Jul 2013 21:37:12 GMT

I seen this problem before. The problem is not with your delim, but with your field names in FIELDS. Change your "-" to "_" or remove "-" entirely. Splunk tends to not like the dash when naming fields.

Example changes:

cs-host to cs_host
cs(User-Agent) to cs_userAgent . Personally I typically use all lower case.
cs(Referer) to cs_referer

Doing this should fix your problem.

Also this simplified props file that works.



[iis_ex]

pulldown_type=true

MAX_TIMESTAMP_LOOKAHEAD=19

TIME_FORMAT=%Y-%m-%d %H:%M:%S

SHOULD_LINEMERGE=false

CHECK_FOR_HEADER=false

TZ=GMT

REPORT-iisw3cfields=iisw3cfields

Hope this help or gets you started cheers.

Re: DELIM does not work

ogdin — Thu, 13 Feb 2014 20:32:30 GMT

If you are using Splunk 6 on both Forwarder and Indexer (or just ingesting logs locally on the indexer) set sourcetype=iis in your inputs.conf.

Under the covers, this is using INDEXED_EXTRACTIONS=W3C in props.conf and will automatically pick up the header and use it for field mappings so you don't have to mess with props and transforms.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime