How to get IIS header extraction to detect change in fields

[yuvalba]

Splunk 5.0.3

I am using the default iis sourcetype for IIS logs, and got iis-2 type created.

I added a new field on the IIS (7.5) server (cs-host) which causes IIS to inserted a new Fields line in the current log file.

The problem is that the change is not detected until a new file is created which created a period of mis-classified data that got the wrong sourcetype (iis-2), later when a new file was created it got the correct new type iis-3

Is this normal behaviour?
Any way to prevent this perdiod of wrong sourcetype detected?

[ogdin]

In Splunk 6, you can use:

http://docs.splunk.com/Documentation/Splunk/latest/Data/Extractfieldsfromfileheadersatindextime

Specifically INDEXED_EXTRACTIONS=W3C for IIS. The only caveat here is that when you change the field format in IIS, say add or remove a field, IIS will write the header mid-file. We won't auto-detect this change (we ignore and won't index any line after the initial header beginning with a #) but when the file rolls (nightly I believe by default), we'll pick up the new header at the top of the new file.

[alacercogitatus · Accepted solution]

With IIS I found the best option was to force the iis sourcetype for all inputs, and have a standard logging in place. By default, I use "all" of the fields. That way, it's the same fields all the time no matter what IIS server is logging. There is a brief cutover time associated with this method, as the extractions don't work for the old sourcetypes, but it simplified the whole "this iis is different than that iis". IIS is IIS for me.

Changing sourcetype mid-file isn't an option, as far as I know. Hence why I standardized the output and make sure it's followed.

[yuvalba]

Thanks.
I also plan to have standard, but I thought the auto header extraction is nice to use. Usually it will be same format.
I prefer to not log all fields due to index volume limits.
Now I have another problem, the latest file is not detected and left as "iis" and having no fields extracted...