Splunk Search

Why isn't this field transform working?

dpanych
Communicator

Hi everyone, I'm having trouble applying the following fields transformation — it's not "parsing" during search time. The regex works fine, it's parsing VALUE, so for example, CODE=22344, but nothing seems to go on during search time. Any help is much appreciated.

Regex:

<([^>]+)>\h+([^<]+)(?:\h+|$)

Sample data:

Jan 22 09:00:00 10.10.0.190 MCS:BS::REPORT::RUN: <Code> 22344 <Type> AUDIT <Severity> PROCESS <Category> SECURITY <User> root <HwSource> MCGUI <Summary> Report ran <report> /Usage Intelligence - Front End Capacity 

Jan 22 09:00:00 10.10.0.190 MCS:BS::REPORT::RUN: <Code> 22344 <Type> AUDIT <Severity> PROCESS <Category> SECURITY <User> root <HwSource> MCGUI <Summary> Report ran <report> /Usage Intelligence - Backend Capacity

Jan 22 08:51:51 10.10.0.190 MCS:AvmgrLoginModule::USER::LOGOFF: <Code> 22343 <Type> AUDIT <Severity> USER <Category> SECURITY <User> MCUser <HwSource> ss-1r <Summary> User logoff successful <action> logout
0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

If you want extract field names and values both from raw data at search time then it is possible using props and transforms.

To accomplish using Splunk Web

  1. Settings -> Fields -> Field Transformations -> New Field Transformation, provide any user friendly name; type regex; copy your regex <([^>]+)>\h+([^<]+)(?:\h+|$) and in format provide $1::$2 and tick Automatically clean field names; Save.
  2. Settings -> Fields -> Field Extractions -> New Field Extraction, provide any user friendly name; select source, host or sourcetype based on your requirement and provide corresponding value in named input; type - select Uses transform; in extraction/Transform provide use friendly name which you have given in step 1 (while creating field transformation); Save

Now search your data with base query and it will automatically extract fields action, category, .... with their values based on regex. I have tested above steps in my lab environment with sample data you have provided and it is extracting various fields with values.

View solution in original post

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

If you want extract field names and values both from raw data at search time then it is possible using props and transforms.

To accomplish using Splunk Web

  1. Settings -> Fields -> Field Transformations -> New Field Transformation, provide any user friendly name; type regex; copy your regex <([^>]+)>\h+([^<]+)(?:\h+|$) and in format provide $1::$2 and tick Automatically clean field names; Save.
  2. Settings -> Fields -> Field Extractions -> New Field Extraction, provide any user friendly name; select source, host or sourcetype based on your requirement and provide corresponding value in named input; type - select Uses transform; in extraction/Transform provide use friendly name which you have given in step 1 (while creating field transformation); Save

Now search your data with base query and it will automatically extract fields action, category, .... with their values based on regex. I have tested above steps in my lab environment with sample data you have provided and it is extracting various fields with values.

dpanych
Communicator

This worked! Thank you very much. I had only the first part done initially.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Search time regex like this (I would assume you are doing it with a rex command) can't extract the field name in this manner. You won't get any fields because you aren't using a named capture group, only unnamed capture groups. If you want this to work, it would have to be done at index time. If you can't do it at index time you need to extract the fields with named capture groups, which is probably going to be a PITA to do with events that are not equivalent to each other. You may have to do something like:

... | rex "\<code\> (?<Code>\d+)" | rex "<\Type\> (?<type>[^\<]+)\s" | rex ...
0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...