I want to add a field to my events that is derived from a discovered field at search time. The new field wil be a primary field used in searches, therefore I can't use "| eval new=substr(..)".
Example logging:
2016-10-18 11:22:33.456 ERROR session=12akeife0wkefs0e835i5l0iwef
When searched for 'ERROR', the field 'session' is automatically discovered. That's good.
Now I want to create a new field 'ses' and fill it with the first 10 characters of 'session'. This is how I think this should be done:
transforms.conf:
[ses]
REGEX=(.{10})
FORMAT=ses::$1
SOURCE_KEY=session
props.conf:
[sourcetype_x]
REPORT-ses = ses
Now when I search for 'ERROR', I would expect to find the new field 'ses' with value '12akeife0w', but it isn't there.
I took notice of:
http://docs.splunk.com/Documentation/Splunk/6.5.0/Admin/Configurationparametersandthedatapipeline#How_configuration_parameters_correlate_to_phases_of_the_pipeline
How configuration parameters correlate to phases of the pipeline - Search phase
http://docs.splunk.com/Documentation/Splunk/6.5.0/admin/propsconf
Search-time field extractions: Why use REPORT if EXTRACT will do?
It's a good question. And much of the time, EXTRACT is all you need for
search-time field extraction. But when you build search-time field
extractions, there are specific cases that require the use of REPORT and the
field transform that it references. Use REPORT if you want to:
...
* Manage formatting of extracted fields, in cases where you are extracting
multiple fields, or are extracting both the field name and field value.
http://docs.splunk.com/Documentation/Splunk/6.5.0/Knowledge/WhenSplunkEnterpriseaddsfields
When field discovery is enabled, Splunk software:
• **Identifies and extracts the first 50 fields that it finds in the event data that match obvious key=value pairs. This 50 field limit is a default that you can modify by editing the [kv] stanza in limits.conf, if you have Splunk Enterprise.
• Extracts any field explicitly mentioned in the search that it might otherwise have found though automatic extraction, but is not among the first 50 fields identified.
• **Performs custom field extractions that you have defined, either through the Field Extractor, the Extracted Fields page in Settings, configuration file edits, or search commands such as rex.
My question(s):
In what phase are discovered fields available?
Are discovered fields available before a transform at search-time?
... View more