All Apps and Add-ons

Help extracting fields

nadine_wondem
New Member

We are having problems with field extractions. The transforms.conf stanza looks like:

[kv_for_cisco_wsa_w3c]
DELIMS = " "
FIELDS = "date", "time", "x_elapsed_time", "cs_username", "c_ip", "s_ip", "cs_bytes", "sc_bytes", "cs_method", "cs_mime_type", "s_computerName", "s_port", "cs_url", "x_webcat_code_full", "x_sophos_virus_name", "x_webroot_threat_name", "x_wbrs_score", "x_wbrs_threat_type", "x_wbrs_threat_reason", "sc_result_code", "sc_result_code_denial", "cs_Referer", "cs_User_Agent"

However, it seems to be confused with cases where there is a quotation mark in the url. It includes part of the x_webcat_code_full as part of the URL. For example the cs_url ends with &dL_inYP=13. This is followed by a space and the x_webcat_code_full which is "Search Engines and Portals"

The app is extracting the cs_url as &dL_inY=555&dL_inYP=13 "Search

It then stores Engines as the x_webcat_code_full and Portals" as the x_webroot_threat_name.

There is a space after the URL so why does it think that the "Search is part of the URL? Any help would be appreciated.

Any idea why this is happening?

0 Karma

jcoates_splunk
Splunk Employee
Splunk Employee

Hi,

the issue is because the data has quotation marks in it.
http://docs.splunk.com/Documentation/Splunk/latest/Admin/transformsconf

DELIMS =
* NOTE: This attribute is only valid for search-time field extractions.
* IMPORTANT: If a value may contain an embedded unescaped double quote
character, such as "foo"bar", use
REGEX, not DELIMS. An escaped double
quote (\") is ok.

This is one of the many reasons why W3C log format is so horrible. Perhaps a SEDCMD on the input to escape those quotes?

0 Karma
Get Updates on the Splunk Community!

CX Day is Coming!

Customer Experience (CX) Day is on October 7th!! We're so excited to bring back another day full of wonderful ...

Strengthen Your Future: A Look Back at Splunk 10 Innovations and .conf25 Highlights!

The Big One: Splunk 10 is Here!  The moment many of you have been waiting for has arrived! We are thrilled to ...

Now Offering the AI Assistant Usage Dashboard in Cloud Monitoring Console

Today, we’re excited to announce the release of a brand new AI assistant usage dashboard in Cloud Monitoring ...