Data Models and incomplete source data

[mjuestel2]

Greetings,

I'm finally tackling the topic of data models within my organization, and am coming across situations I am needing to solve for.

1. Windows authentication data which has a null values in the src field, due to the type of authentication taking place.

I understand that field aliasing comes into play, and I tried that - however, I tried aliasing a calculated field, which does not work of course. Now, I am having to go back to see if there is another field I can alias instead.

I guess my ask with this post here is to get some strategies from other Splunk users who have tackled data cleanup and data models.

Are null values acceptable for certain situations? Or, must every required data model field be complete? Such as action, app, dest, src, user etc.?

I appreciate some feedback regarding this topic.

[VatsalJagani · Accepted solution]

@mjuestel2 - I see you mentioned the calculated field (props.conf EVAL) did not work. This is possible because of props.conf parameter executes in a certain order. One more thing, EVALs execute in parallel so if the initial field is getting extracted by EVAL, you cannot use the new field to generate another calculated field with EVAL.

Reference - https://docs.splunk.com/Documentation/Splunk/latest/Knowledge/Searchtimeoperationssequence 

- https://community.splunk.com/t5/Getting-Data-In/Why-is-eval-in-props-conf-not-performing-in-a-partic...

Regarding null values, an easier solution to implement is with data-model level eval.

Below is an example of the src field in one of the CIM data-model. This can be applied to any data-model and any field.

I hope this helps!! Kindly upvote.

[mjuestel2]

Thanks for your reply... not sure how your response will be help me. The Windows TA is already filling-in "null" for the src field.

I was wondering if that is a valid result for a data model? 

I'm looking at this from an investigative perspective - if a Security Analyst were to pull up the auth data model and look for failed logins, and sees events with a src of null.  They would only know the dest server, user etc. - and not be able to see where these authentication events originated from. 

That is what I am trying to better understand... is there a way to "complete the picture"? A best practice around this?

[VatsalJagani]

@mjuestel2 - If the raw event itself does not have proper field value, then the data-model will not have value as well which is fine. I don't understand why do you expects the data-model to have src value while the system itself has not generated the value of the field.

[mjuestel2]

This is what I wanted to confirm... Again, I'm new to this whole data model thing - and was looking for best practices around some of these issues I've been coming across.

Thanks for getting back with me!