topic Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis? in Getting Data In

So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

hoopydave — Mon, 28 Sep 2020 18:03:21 GMT

If I add INDEXED_EXTRACTIONS = w3c using a sourcetype other than iis, it does not work for defining the field names. Is there a .conf file that I can define a different sourcetype for this functionality? We have several groups with different IIS logs and I'd like to call the sourcetype iis_group1, iis_group2, etc.

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

vasanthmss — Thu, 30 Oct 2014 23:45:49 GMT

Create a different stanzas in the props.conf and use your stanzas (sourcetype) while indexing the log

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

hoopydave — Mon, 28 Sep 2020 18:03:29 GMT

I have created different stanzas using different sourcetypes, but the only time I can get the field extractions to work correctly is if I use [iss] as the sourcetype. If i use something like [iss_group1], the field names do not get extracted correctly. Running the same query (index=dotnet) see the examples below:

This one works using [iis] as the sourcetype:
Inputs.conf

    # DOTNET IIS LOGS
    [monitor://D:\Applications\*\*\W*\*.log]
    sourcetype = iis
    index = dotnet
    ignoreOlderThan = 5d
    disabled = false

props.conf

    [iis]
    INDEXED_EXTRACTIONS = w3c

Returns:

    Interesting Fields
        a c_ip 1
        # cs_bytes 100+
        a cs_method 2
        a cs_uri_query 25
        a cs_uri_stem 86
        a cs_User_Agent 4
        a cs_username 3
        a date 1

If I change that sourcetype to [iis_group1] and use INDEXED_EXTRACTIONS = w3c, Splunk does not properly extract the field names:

inputs.conf:

    # DOTNET IIS LOGS
    [monitor://D:\Applications\*\*\W*\*.log]
    sourcetype = iis_group1
    index = dotnet
    ignoreOlderThan = 5d
    disabled = false

props.conf

    [iis_group1]
    INDEXED_EXTRACTIONS = w3c

Returns

Interesting Fields

    # date_hour 24
    # date_mday 5
    # date_minute 60
    a date_month 1
    # date_second 60
    a date_wday 5
    # date_year 1
    a date_zone 1

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

somesoni2 — Fri, 31 Oct 2014 15:04:56 GMT

Give this a try

[iis_group1]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
INDEXED_EXTRACTIONS = w3c
detect_trailing_nulls = auto
category = Web

Update

Try this

[iis_group1]
pulldown_type = true 
MAX_TIMESTAMP_LOOKAHEAD = 32
SHOULD_LINEMERGE = False
FIELD_DELIMITER = whitespace
FIELD_HEADER_REGEX = ^#Fields:\\s*(.*)
MISSING_VALUE_REGEX = -
TIME_FORMAT = %Y-%m-%d %H:%M:%S
TIMESTAMP_FIELDS = date,time

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

hoopydave — Mon, 28 Sep 2020 18:03:37 GMT

Thanks for your response, somesoni2,
As suggested, I tried:
inputs.conf

# DOTNET IIS LOGS
[monitor://D:\Applications\*\*\W*\*.log]
sourcetype = iis_group1
index = dotnet
ignoreOlderThan = 5d
disabled = false

props.conf

        [iis_group1]
        pulldown_type = true
        MAX_TIMESTAMP_LOOKAHEAD = 32
        SHOULD_LINEMERGE = False
        INDEXED_EXTRACTIONS = w3c
        detect_trailing_nulls = auto
        category = Web

That makes the data look good, but it is still not pulling in the standard IIS field names; c_ip, cs_bytes, cs_method, cs_uri_query, cs_uri_stem, cs_User_Agent, cs_username, etc

Results:

    Interesting Fields

        a d 15
        # date_hour 1
        # date_mday 1
        # date_minute 1
        a date_month 1
        # date_second 12
        a date_wday 1

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

jnv7 — Tue, 08 Sep 2015 13:23:04 GMT

hi, this is an old topic but I just found this exact same behavior. Has anyone ever found a justification for this? If I use sourcetype=iis, the w3c fields are automatically extracted. On the other hand, if I copy the [iis] stanza into my props.conf with another name and use this as the sourcetype, the fields are not extracted anymore.

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

jnv7 — Tue, 29 Sep 2020 07:14:35 GMT

Well, just in case anyone bumps into this, I guess it was quite a newbie problem. I managed to do it by deploying to the forwarder (instead of the indexer) a props.conf file with a copy of the [iis] default stanza but with the different name [iis_group1] in the example. This way the w3c fields in iis_group1 should be automatically extracted.

Re: So can you use INDEXED_EXTRACTIONS = w3c with something other than the sourcetype of iis?

rajbir1 — Thu, 17 Mar 2016 14:33:33 GMT

Make sure you are sending a copy of your custom props.conf to the UF as well. I had the same issue but it started working when I put a copy of props.conf with my custom sourcetype on UF's.