All Apps and Add-ons

Help extracting fields

New Member

We are having problems with field extractions. The transforms.conf stanza looks like:

[kv_for_cisco_wsa_w3c]
DELIMS = " "
FIELDS = "date", "time", "x_elapsed_time", "cs_username", "c_ip", "s_ip", "cs_bytes", "sc_bytes", "cs_method", "cs_mime_type", "s_computerName", "s_port", "cs_url", "x_webcat_code_full", "x_sophos_virus_name", "x_webroot_threat_name", "x_wbrs_score", "x_wbrs_threat_type", "x_wbrs_threat_reason", "sc_result_code", "sc_result_code_denial", "cs_Referer", "cs_User_Agent"

However, it seems to be confused with cases where there is a quotation mark in the url. It includes part of the x_webcat_code_full as part of the URL. For example the cs_url ends with &dL_inYP=13. This is followed by a space and the x_webcat_code_full which is "Search Engines and Portals"

The app is extracting the cs_url as &dL_inY=555&dL_inYP=13 "Search

It then stores Engines as the x_webcat_code_full and Portals" as the x_webroot_threat_name.

There is a space after the URL so why does it think that the "Search is part of the URL? Any help would be appreciated.

Any idea why this is happening?

0 Karma

Splunk Employee
Splunk Employee

Hi,

the issue is because the data has quotation marks in it.
http://docs.splunk.com/Documentation/Splunk/latest/Admin/transformsconf

DELIMS =
* NOTE: This attribute is only valid for search-time field extractions.
* IMPORTANT: If a value may contain an embedded unescaped double quote
character, such as "foo"bar", use
REGEX, not DELIMS. An escaped double
quote (\") is ok.

This is one of the many reasons why W3C log format is so horrible. Perhaps a SEDCMD on the input to escape those quotes?

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!