I have a set of logs that require a pretty complex set of regexes to parse. The data has about 8 columns separated by commas, but the values have commas all over the place too so it's not a simple CSV extraction. To make it worse, each column has a bunch of different multivalue field/value pairs with spaces, double-quotes, commas, all sorts of stuff.
In any case, I have successfully extracted the columns into fields with an EXTRACT in props. Then I use that field as the SOURCE_KEY in transforms.conf to do additional extractions - schematically like what I have below.
The data in the COLUMN3 field is not meaningful to the user - it is only used as a simpler means to extract other fields. Therefore I don't want the COLUMN3 field to remain as a search-time extraction for the user. Is there a way to delete the COLUMN3 field after all the fields have been extracted from it?
props.conf
[logs]
EXTRACT-COLUMN1,COLUMN2,COLUMN3 = ^(.*?),(.*?),(.*?)$
REPORT-field_from_COLUMN3 = field_from_COLUMN3
transforms.conf
[field_from_COLUMN3]
SOURCE_KEY = COLUMN3
REGEX = Field=(?P<Field>.*)
