Automatic field extraction syntax: quoting, escaping quotes

[larrywest]

(Note that this entire post is about text being written to logs that Splunk scans, not about queries or query syntax.)

I would like to know what syntax (regex?) Splunk uses for its automatic field extraction.

The obvious rule is that if the field has embedded spaces or certain other characters, it has to be quoted, with double quotes (never single quotes/apostrophes; I haven't tried UNICODE puncuation very much).

For example, I've learned that a "|" terminates a field as far as Splunk auto-extraction is concern, and so any field with a "|" must be quoted.

Is there a list of such rules, a regular expression, or code available to inspect?

Here's a practical example: the 'weak' form of the HTTP ETags (or If-None-Match) header look like this:

  Etag: W/"some-identifying-text"

Now if I want to log that, I need to be careful not to quote it, because then Splunk will see all these values as having the value "W/".

Or, I've found -- by trial and error -- that I can escape quotes by doubling them. E.g.,

  Etag: "W/""some-identifying-text"

But it would be far far easier to know rather than guess the syntax rules. Can someone point me towards them?

Also, it would be good to know which of the UNICODE quoting characters Splunk inteprets as such -- e.g., it should probably handle the "Ps", "Pe", "Pi", and "Pf" categories as described in http://www.unicode.org/versions/Unicode9.0.0/ch04.pdf, but does it? If not, what characters does it consider to be quoting characters, other than ASCII double-quote (")?

[Richfez]

For how Splunk breaks up streams of "stuff" you'll want to refer to segmenters.conf

Here's a link to the configuration file for segmenters.conf. You may also fine it useful to read more explanation about segmentation and segmentation types. Those two links should give you what you need - a list of which characters do what things by default when Splunk breaks up events into fields and segments.