I have a data source where the log format is the same but one attribute changes for various logs. I want to extract the field name and field value from the log itself.. is it possible? Please find sample logs below
May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" whencreated=""2019-05-16T08:31:32.0000000Z""
May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" pwdlastset=""2019-05-16T08:31:32.0000000Z""
May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" badpwdcount="20"
May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" operatingsystemversion=""6.1 (7601)""
If you notice, the last attribute alone changes for each log. I want to extract the fields like mentioned below.
field value
whencreated 2019-05-16T08:31:32.0000000Z
pwdlastset 2019-05-16T08:31:32.0000000Z
badpwdcount 20
operatingsystemversion 6.1 (7601)
Hello @anubhp
in transforms.conf
[extract_last_field]
REGEX =\s([^ =]+)="+([^="]*)"+\s*$
FORMAT = $1::$2
in props.conf
[mysourcetype]
REPORT-extract_last_field = extract_last_field
what does it do? The regex extracts the last field with the value separated by the equal sign just before the end of the line.
check it here: https://regex101.com/r/d3dv5l/1
Hi,
it is not clear, if you want to index all data and extract the fields at search time by adding some field extraction to your sourcetype or if you want to reduce the data indexed by only indexing the unique field.
Oliver
I am able to extract all the fields in the logs except the last one. The last attribute is a key value pair where i want the key as the field name and value as the field value as shown in the example