Create Props Stanzas Based on Field Value

[tom_porter]

I have events that return different structured fields depending on the value of a field called TYPE.  This all comes from the same sourcetype.  For example:

if type=TYPE1, I might have fields called: TYPE1.exe, TYPE1.comm, TYPE1.path, TYPE1.filename

if type=TYPE2, I might have fields called: TYPE2.comm, TYPE2.path, TYPE2.host

As you can see, each type brings a different set of base fields.  We are using data model searches so I want to get these base fields into CIM compliance.   Is there a way to create stanzas in props.conf or transforms.conf that will allow me to field alias these values based on the type value?  I tried straight-out field aliasing in props.conf only to find I was actually overwriting values due to precedence/order of my field alias commands.

Thanks in advance,

[gcusello · Accepted solution]

Hi @tom_porter,

using CIM you have two solutions:

you could add all the fields to the CIM data Model (I don't like),

you could try to normalize your data adding few fields and using calculated fields to insert the correct values.

For example you could add some field to the CIM data Model (exe, comm, path, filename hostname) and then create some calculated fields:

| eval 
   exe=if(type=TYPE1, TYPE1.exe, TYPE2.exe),
   comm=if(type=TYPE1, TYPE1.comm, TYPE2.comm)

then you can use thee fields in your searches using Data Model values.

For more infos about normalization see:

https://www.splunk.com/en_us/blog/learn/data-normalization.html?locale=en_us

https://docs.splunk.com/Documentation/CIM/5.2.0/User/UsetheCIMtonormalizedataatsearchtime 

Ciao.

Giuseppe

[tom_porter]

Thank you...will try it out this week.  We actually have 40 unique type values (ouch), so this may take a while.