Splunk Search

Why isn't this field transform working?

dpanych
Communicator

Hi everyone, I'm having trouble applying the following fields transformation — it's not "parsing" during search time. The regex works fine, it's parsing VALUE, so for example, CODE=22344, but nothing seems to go on during search time. Any help is much appreciated.

Regex:

<([^>]+)>\h+([^<]+)(?:\h+|$)

Sample data:

Jan 22 09:00:00 10.10.0.190 MCS:BS::REPORT::RUN: <Code> 22344 <Type> AUDIT <Severity> PROCESS <Category> SECURITY <User> root <HwSource> MCGUI <Summary> Report ran <report> /Usage Intelligence - Front End Capacity 

Jan 22 09:00:00 10.10.0.190 MCS:BS::REPORT::RUN: <Code> 22344 <Type> AUDIT <Severity> PROCESS <Category> SECURITY <User> root <HwSource> MCGUI <Summary> Report ran <report> /Usage Intelligence - Backend Capacity

Jan 22 08:51:51 10.10.0.190 MCS:AvmgrLoginModule::USER::LOGOFF: <Code> 22343 <Type> AUDIT <Severity> USER <Category> SECURITY <User> MCUser <HwSource> ss-1r <Summary> User logoff successful <action> logout
0 Karma
1 Solution

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

If you want extract field names and values both from raw data at search time then it is possible using props and transforms.

To accomplish using Splunk Web

  1. Settings -> Fields -> Field Transformations -> New Field Transformation, provide any user friendly name; type regex; copy your regex <([^>]+)>\h+([^<]+)(?:\h+|$) and in format provide $1::$2 and tick Automatically clean field names; Save.
  2. Settings -> Fields -> Field Extractions -> New Field Extraction, provide any user friendly name; select source, host or sourcetype based on your requirement and provide corresponding value in named input; type - select Uses transform; in extraction/Transform provide use friendly name which you have given in step 1 (while creating field transformation); Save

Now search your data with base query and it will automatically extract fields action, category, .... with their values based on regex. I have tested above steps in my lab environment with sample data you have provided and it is extracting various fields with values.

View solution in original post

harsmarvania57
SplunkTrust
SplunkTrust

Hi,

If you want extract field names and values both from raw data at search time then it is possible using props and transforms.

To accomplish using Splunk Web

  1. Settings -> Fields -> Field Transformations -> New Field Transformation, provide any user friendly name; type regex; copy your regex <([^>]+)>\h+([^<]+)(?:\h+|$) and in format provide $1::$2 and tick Automatically clean field names; Save.
  2. Settings -> Fields -> Field Extractions -> New Field Extraction, provide any user friendly name; select source, host or sourcetype based on your requirement and provide corresponding value in named input; type - select Uses transform; in extraction/Transform provide use friendly name which you have given in step 1 (while creating field transformation); Save

Now search your data with base query and it will automatically extract fields action, category, .... with their values based on regex. I have tested above steps in my lab environment with sample data you have provided and it is extracting various fields with values.

dpanych
Communicator

This worked! Thank you very much. I had only the first part done initially.

0 Karma

cpetterborg
SplunkTrust
SplunkTrust

Search time regex like this (I would assume you are doing it with a rex command) can't extract the field name in this manner. You won't get any fields because you aren't using a named capture group, only unnamed capture groups. If you want this to work, it would have to be done at index time. If you can't do it at index time you need to extract the fields with named capture groups, which is probably going to be a PITA to do with events that are not equivalent to each other. You may have to do something like:

... | rex "\<code\> (?<Code>\d+)" | rex "<\Type\> (?<type>[^\<]+)\s" | rex ...
0 Karma
Get Updates on the Splunk Community!

What's new in Splunk Cloud Platform 9.1.2312?

Hi Splunky people! We are excited to share the newest updates in Splunk Cloud Platform 9.1.2312! Analysts can ...

What’s New in Splunk Security Essentials 3.8.0?

Splunk Security Essentials (SSE) is an app that can amplify the power of your existing Splunk Cloud Platform, ...

Let’s Get You Certified – Vegas-Style at .conf24

Are you ready to level up your Splunk game? Then, let’s get you certified live at .conf24 – our annual user ...