Help extracting fields

[nadine_wondem]

We are having problems with field extractions. The transforms.conf stanza looks like:

[kv_for_cisco_wsa_w3c]
DELIMS = " "
FIELDS = "date", "time", "x_elapsed_time", "cs_username", "c_ip", "s_ip", "cs_bytes", "sc_bytes", "cs_method", "cs_mime_type", "s_computerName", "s_port", "cs_url", "x_webcat_code_full", "x_sophos_virus_name", "x_webroot_threat_name", "x_wbrs_score", "x_wbrs_threat_type", "x_wbrs_threat_reason", "sc_result_code", "sc_result_code_denial", "cs_Referer", "cs_User_Agent"

However, it seems to be confused with cases where there is a quotation mark in the url. It includes part of the x_webcat_code_full as part of the URL. For example the cs_url ends with &dL_inYP=13. This is followed by a space and the x_webcat_code_full which is "Search Engines and Portals"

The app is extracting the cs_url as &dL_inY=555&dL_inYP=13 "Search

It then stores Engines as the x_webcat_code_full and Portals" as the x_webroot_threat_name.

There is a space after the URL so why does it think that the "Search is part of the URL? Any help would be appreciated.

Any idea why this is happening?

[jcoates_splunk]

Hi,

the issue is because the data has quotation marks in it.
http://docs.splunk.com/Documentation/Splunk/latest/Admin/transformsconf

DELIMS =
* NOTE: This attribute is only valid for search-time field extractions.
* IMPORTANT: If a value may contain an embedded unescaped double quote
character, such as "foo"bar", use
REGEX, not DELIMS. An escaped double
quote (\") is ok.

This is one of the many reasons why W3C log format is so horrible. Perhaps a SEDCMD on the input to escape those quotes?