Is it possible to extract the field name itself from the logs

[anubhp]

I have a data source where the log format is the same but one attribute changes for various logs. I want to extract the field name and field value from the log itself.. is it possible? Please find sample logs below

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" whencreated=""2019-05-16T08:31:32.0000000Z""

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" pwdlastset=""2019-05-16T08:31:32.0000000Z""

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" badpwdcount="20"

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" operatingsystemversion=""6.1 (7601)""

If you notice, the last attribute alone changes for each log. I want to extract the fields like mentioned below.

field value

whencreated 2019-05-16T08:31:32.0000000Z
pwdlastset 2019-05-16T08:31:32.0000000Z
badpwdcount 20
operatingsystemversion 6.1 (7601)

[PavelP]

Hello @anubhp

in transforms.conf

[extract_last_field]
REGEX =\s([^ =]+)="+([^="]*)"+\s*$
FORMAT = $1::$2

in props.conf

[mysourcetype]
REPORT-extract_last_field = extract_last_field

what does it do? The regex extracts the last field with the value separated by the equal sign just before the end of the line.

check it here: https://regex101.com/r/d3dv5l/1

[ololdach]

Hi,
it is not clear, if you want to index all data and extract the fields at search time by adding some field extraction to your sourcetype or if you want to reduce the data indexed by only indexing the unique field.
Oliver

[anubhp]

I am able to extract all the fields in the logs except the last one. The last attribute is a key value pair where i want the key as the field name and value as the field value as shown in the example