Splunk Search

Delete a search-time field after extraction

_smp_
Builder

I have a set of logs that require a pretty complex set of regexes to parse. The data has about 8 columns separated by commas, but the values have commas all over the place too so it's not a simple CSV extraction. To make it worse, each column has a bunch of different multivalue field/value pairs with spaces, double-quotes, commas, all sorts of stuff.

In any case, I have successfully extracted the columns into fields with an EXTRACT in props. Then I use that field as the SOURCE_KEY in transforms.conf to do additional extractions - schematically like what I have below.

The data in the COLUMN3 field is not meaningful to the user - it is only used as a simpler means to extract other fields. Therefore I don't want the COLUMN3 field to remain as a search-time extraction for the user. Is there a way to delete the COLUMN3 field after all the fields have been extracted from it?

props.conf
[logs]
EXTRACT-COLUMN1,COLUMN2,COLUMN3 = ^(.*?),(.*?),(.*?)$
REPORT-field_from_COLUMN3 = field_from_COLUMN3

transforms.conf
[field_from_COLUMN3]
SOURCE_KEY = COLUMN3
REGEX = Field=(?P<Field>.*)

0 Karma
1 Solution

martin_mueller
SplunkTrust
SplunkTrust

You should be able to define a calculated field EVAL-COLUMN3 = null() that overwrites the value after the transforms ran: https://docs.splunk.com/Documentation/Splunk/7.0.2/Knowledge/Searchtimeoperationssequence

View solution in original post

martin_mueller
SplunkTrust
SplunkTrust

You should be able to define a calculated field EVAL-COLUMN3 = null() that overwrites the value after the transforms ran: https://docs.splunk.com/Documentation/Splunk/7.0.2/Knowledge/Searchtimeoperationssequence

View solution in original post

_smp_
Builder

Brilliant! Thanks!

0 Karma
Register for .conf21 Now! Go Vegas or Go Virtual!

How will you .conf21? You decide! Go in-person in Las Vegas, 10/18-10/21, or go online with .conf21 Virtual, 10/19-10/20.