Knowledge Management

Create Props Stanzas Based on Field Value

tom_porter
Explorer

I have events that return different structured fields depending on the value of a field called TYPE.  This all comes from the same sourcetype.  For example:

if type=TYPE1, I might have fields called: TYPE1.exe, TYPE1.comm, TYPE1.path, TYPE1.filename

if type=TYPE2, I might have fields called: TYPE2.comm, TYPE2.path, TYPE2.host

As you can see, each type brings a different set of base fields.  We are using data model searches so I want to get these base fields into CIM compliance.   Is there a way to create stanzas in props.conf or transforms.conf that will allow me to field alias these values based on the type value?  I tried straight-out field aliasing in props.conf only to find I was actually overwriting values due to precedence/order of my field alias commands.

Thanks in advance,

Labels (3)
0 Karma
1 Solution

gcusello
SplunkTrust
SplunkTrust

Hi @tom_porter,

using CIM you have two solutions:

you could add all the fields to the CIM data Model (I don't like),

you could try to normalize your data adding few fields and using calculated fields to insert the correct values.

For example you could add some field to the CIM data Model (exe, comm, path, filename hostname) and then create some calculated fields:

| eval 
   exe=if(type=TYPE1, TYPE1.exe, TYPE2.exe),
   comm=if(type=TYPE1, TYPE1.comm, TYPE2.comm)

then you can use thee fields in your searches using Data Model values.

For more infos about normalization see:

https://www.splunk.com/en_us/blog/learn/data-normalization.html?locale=en_us

https://docs.splunk.com/Documentation/CIM/5.2.0/User/UsetheCIMtonormalizedataatsearchtime 

Ciao.

Giuseppe

View solution in original post

gcusello
SplunkTrust
SplunkTrust

Hi @tom_porter,

using CIM you have two solutions:

you could add all the fields to the CIM data Model (I don't like),

you could try to normalize your data adding few fields and using calculated fields to insert the correct values.

For example you could add some field to the CIM data Model (exe, comm, path, filename hostname) and then create some calculated fields:

| eval 
   exe=if(type=TYPE1, TYPE1.exe, TYPE2.exe),
   comm=if(type=TYPE1, TYPE1.comm, TYPE2.comm)

then you can use thee fields in your searches using Data Model values.

For more infos about normalization see:

https://www.splunk.com/en_us/blog/learn/data-normalization.html?locale=en_us

https://docs.splunk.com/Documentation/CIM/5.2.0/User/UsetheCIMtonormalizedataatsearchtime 

Ciao.

Giuseppe

tom_porter
Explorer

Thank you...will try it out this week.  We actually have 40 unique type values (ouch), so this may take a while.

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In September, the Splunk Threat Research Team had two releases of new security content via the Enterprise ...

New in Observability - Improvements to Custom Metrics SLOs, Log Observer Connect & ...

The latest enhancements to the Splunk observability portfolio deliver improved SLO management accuracy, better ...

Improve Data Pipelines Using Splunk Data Management

  Register Now   This Tech Talk will explore the pipeline management offerings Edge Processor and Ingest ...