Splunk Search

Is it possible to extract the field name itself from the logs

anubhp
New Member

I have a data source where the log format is the same but one attribute changes for various logs. I want to extract the field name and field value from the log itself.. is it possible? Please find sample logs below

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" whencreated=""2019-05-16T08:31:32.0000000Z""

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" pwdlastset=""2019-05-16T08:31:32.0000000Z""

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" badpwdcount="20"

May 15 04:29:41 host datasource: "0" "Enterprise Forest" "domain" "field2" "severity" "user" "id" "profileid" "type" "eventid" operatingsystemversion=""6.1 (7601)""

If you notice, the last attribute alone changes for each log. I want to extract the fields like mentioned below.

field value

whencreated 2019-05-16T08:31:32.0000000Z
pwdlastset 2019-05-16T08:31:32.0000000Z
badpwdcount 20
operatingsystemversion 6.1 (7601)

0 Karma

PavelP
Motivator

Hello @anubhp

in transforms.conf

[extract_last_field]
REGEX =\s([^ =]+)="+([^="]*)"+\s*$
FORMAT = $1::$2

in props.conf

[mysourcetype]
REPORT-extract_last_field = extract_last_field

what does it do? The regex extracts the last field with the value separated by the equal sign just before the end of the line.

check it here: https://regex101.com/r/d3dv5l/1

0 Karma

ololdach
Builder

Hi,
it is not clear, if you want to index all data and extract the fields at search time by adding some field extraction to your sourcetype or if you want to reduce the data indexed by only indexing the unique field.
Oliver

0 Karma

anubhp
New Member

I am able to extract all the fields in the logs except the last one. The last attribute is a key value pair where i want the key as the field name and value as the field value as shown in the example

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...