Splunk Search

In a field extraction, why does Splunk remove leading and trailing whitespace?

kritho
Explorer

Hi folks,
I have a freeradius log authenticating wifi-connections. The field extractions looks fine with my favourite regex tester (which returns 4 different values), however when Splunk extracts the fields, leading and trailing spaces is removed within the "user" field, and only one value for user is extracted.
example logs for user rodney:

Mon Jun 19 10:33:54 2017 : Auth: (285089) Invalid user: [rodney ] (from client wifi-controller-1 port 13 cli 8c-f5-a3-e1-65-f1)
Mon Jun 19 10:33:54 2017 : Auth: (285089) Invalid user: [ rodney] (from client wifi-controller-1 port 13 cli 8c-f5-a3-e1-65-f1)
Mon Jun 19 10:33:54 2017 : Auth: (285089) Invalid user: [ rodney ] (from client wifi-controller-1 port 13 cli 8c-f5-a3-e1-65-f1)
Mon Jun 19 10:33:54 2017 : Auth: (285089) Invalid user: [rodney] (from client wifi-controller-1 port 13 cli 8c-f5-a3-e1-65-f1)

Notice four different values of user within brackets. The following regex gives only one user value: rodney

(?P<time_stamp>\w++\s++\w++\s++\d++\s++\d{2}\:\d{2}\:\d{2}\s++\d{4})\s++\:\s++(?P<process>[^:]*)\:\s++(?P<process_id>[^\s]*)\s++(?P<action>[^:]*)\:\s++(?P<message>\[(?P<user>[^]]*|\s+)\]\s++\(\w++\s++\w++\s++(?P<client>[^\s]*)\s++\w++\s++(?P<port>[^\s]*)\s++\w++\s++(?P<mac_address>[^)|\s]*)(\s++\w++(?P<via>[^)]*))?[^\s|\n|\r]*)

Why am I only getting one user value in Splunk?`

Any takers?
Brgds
Kristen

kritho
Explorer

Yes,
I just tried the setting, and theres no change. (still stripping leading/trailing Spaces after setting KV_TRIM_SPACES to true)

k

0 Karma

somesoni2
Revered Legend

See this (http://docs.splunk.com/Documentation/Splunk/6.6.1/Admin/Propsconf#Field_extraction_configuration, lookup for the key)

KV_TRIM_SPACES = true|false
* Modifies the behavior of KV_MODE when set to auto, and auto_escaped.
* Traditionally, automatically identified fields have leading and trailing
  whitespace removed from their values.
  * Example event: 2014-04-04 10:10:45 myfield=" apples "
    would result in a field called 'myfield' with a value of 'apples'.
* If this value is set to false, then external whitespace then this outer
  space is retained.
  * Example: 2014-04-04 10:10:45 myfield=" apples "
    would result in a field called 'myfield' with a value of ' apples '.
* The trimming logic applies only to space characters, not tabs, or other
  whitespace.
* NOTE: The Splunk UI currently has limitations with displaying and
  interactively clicking on  fields that have leading or trailing
  whitespace.  Field values with leading or trailing spaces may not look
  distinct in the event viewer, and clicking on a field value will typically
  insert the term into the search string without its embedded spaces.
  * These warts are not specific to this feature.  Any such embedded spaces
    will behave this way.
  * The Splunk search language and included commands will respect the spaces.
* Defaults to true.

micahkemp
Champion

The solution described is described as being specific to KV_MODE=auto (or auto_escaped). I have had this same issue with props/transforms-defined extractions. I haven't attempted setting this field and seeing if it changes the behavior I've seen, though.

0 Karma
Get Updates on the Splunk Community!

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...

3-2-1 Go! How Fast Can You Debug Microservices with Observability Cloud?

Register Join this Tech Talk to learn how unique features like Service Centric Views, Tag Spotlight, and ...