DELIM does not work

[wsnyder2]

We can not get field extraction to work with IIS log files. Any suggestions?

transforms.conf
[iisw3cfields]
DELIMS = " "
FIELDS = date,time,s-computername,s-ip,cs-method,cs-uri-stem,cs-uri-query,s-port,cs-username,c-ip,cs-version,cs(User-Agent),cs(Cookie),cs(Referer),cs-host,sc-status,sc-bytes,cs-bytes,time-taken

props.conf (relevant part)
...
[iis_ex]
TIME_PREFIX = ^
MAX_TIMESTAMP_LOOKAHEAD = 19
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TZ = GMT
LINE_BREAKER = ([\r\n]+)(\d{4}-\d{2}-\d{2}\s\d{2}:\d{2}:\d{2})
SHOULD_LINEMERGE = False
TRUNCATE = 10000
KV_MODE = None
pulldown_type = true
REPORT-iisw3cfields = iisw3cfields

Here are a few lines from our log file ...

Software: Microsoft Internet Information Services 6.0

Version: 1.0

Date: 2013-07-07 00:00:00

Fields: date time s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-bytes cs-bytes time-taken

2013-07-06 23:59:59 TLOWPN29 192.168.122.82 GET /secure/EmployeeHome.aspx - 443 - 207.190.231.139 Mozilla/4.0+(compatible;+MSIE+7.0;+Windows+NT+5.2;+WOW64;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729) training=true;+SessionGUID=ee81cbbf-9da8-4532-92ae-24a4e8ad6147;+StyleSheetTheme=pxxxx;+SSODeferral=true hxxx://xxxxx.xxxx.com/secure/login.aspx?alias=0414T735 xxxxxxx.xxxxx.com 200 68499 649 1109

2013-07-07 00:00:00 TLOWPN29 192.168.122.82 GET /secure/Images/exception.gif - 443 - 38.102.232.83 Mozilla/4.0+(compatible;+MSIE+8.0;+Windows+NT+5.1;+Trident/4.0;+.NET+CLR+1.1.4322;+.NET+CLR+2.0.50727;+.NET+CLR+3.0.4506.2152;+.NET+CLR+3.5.30729;+.NET4.0C;+.NET4.0E) StyleSheetTheme=pxxxx;+SessionGUID=00000000-0000-0000-0000-000000000000;+training=true hxxps://xxxxxxx.xxxx.com/secure/login.aspx tixxxxxx.xxxxx.com 200 1338 597 15

[ogdin]

If you are using Splunk 6 on both Forwarder and Indexer (or just ingesting logs locally on the indexer) set sourcetype=iis in your inputs.conf.

Under the covers, this is using INDEXED_EXTRACTIONS=W3C in props.conf and will automatically pick up the header and use it for field mappings so you don't have to mess with props and transforms.

http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

[bmacias84]

I seen this problem before. The problem is not with your delim, but with your field names in FIELDS. Change your "-" to "_" or remove "-" entirely. Splunk tends to not like the dash when naming fields.

Example changes:

• cs-host to cs_host
• cs(User-Agent) to cs_userAgent . Personally I typically use all lower case.
• cs(Referer) to cs_referer

Doing this should fix your problem.

Also this simplified props file that works.

[iis_ex]

pulldown_type=true

MAX_TIMESTAMP_LOOKAHEAD=19

TIME_FORMAT=%Y-%m-%d %H:%M:%S

SHOULD_LINEMERGE=false

CHECK_FOR_HEADER=false

TZ=GMT

REPORT-iisw3cfields=iisw3cfields

Hope this help or gets you started cheers.

[linu1988]

i think it's overcooked. Let it flow into splunk. Splunk should be able to get the events divided whereever it finds time at the start. After that, do the field extraction on the UI, which is easier, where you can actually see the matching. If the time is not correct, set the Props.conf for time recognition. Thanks