Getting Data In

Possible to extract same value from different fields in props.conf?

Motivator

For Exchange message trace logs I am extracting the user as following in the props.conf file:

EXTRACT-user = "RecipientAddress":"(?<user>\S+)@

I would also like to extract the user from the SenderAddress as follows:

EXTRACT-user = "SenderAddress":"(?<user>\S+)@

Can I extract from two different fields for the user field, or will this potentially cause confusion?

Thx

0 Karma
1 Solution

Esteemed Legend

I would do it like this:

EXTRACT-user = "RecipientAddress":"(?<user_recipient>\S+)@
EXTRACT-user = "SenderAddress":"(?<user_sender>\S+)@
EVAL-user = mvappend(user_recipient, user_sender)

View solution in original post

0 Karma

Esteemed Legend

I would do it like this:

EXTRACT-user = "RecipientAddress":"(?<user_recipient>\S+)@
EXTRACT-user = "SenderAddress":"(?<user_sender>\S+)@
EVAL-user = mvappend(user_recipient, user_sender)

View solution in original post

0 Karma

Motivator

Thx for the extracts.

My dashboard is currently set up to use 'user' as the value as my Shib and Duo logs have the user field. My message trace logs does not have the 'user' field, but 'src_user' instead. If I set the extracts you graciously provided, will this nullify the Email CIM data model of using 'src_user'?

Thx

0 Karma

Esteemed Legend

You are free to change EVAL-user to EVAL-src_user or EVAL-some_other_field_name and it will work that way. If you keep it as-is, it will override any existing value of user for this sourcetype.

If you would like to prefer any existing value, then use this:

 EVAL-user = coalesce(user, mvappend(user_recipient, user_sender))

If you would like to keep both any existing value and this new value, then use this:

EVAL-user = mvappend(user, user_recipient, user_sender)

This covers all possible desires. Be sure to click Accept to close the question.

0 Karma

Motivator

Thx for the various breakdowns

0 Karma

Ultra Champion

It might make more sense to put one in src_user and the other in user. Or better yet, have a look at the definition for the CIM Email data model: https://docs.splunk.com/Documentation/CIM/4.13.0/User/Email and adhere to that.

If you really want to generate a multi-valued field user, with both sender and recipient(s), extract each into a separate field (e.g. s_user and r_user) and then do something like:

EVAL-user = mvappend(s_user,r_user)

Or use a REPORT extraction like this:

props.conf

REPORT-user = exchange-user

transforms.conf

[exchange-user]
REGEX = "(?:RecipientAddress|SenderAddress)":"(?<user>\S+)@
MV_ADD = true

Motivator

Frank,

Thx for the reply and info.

What I'm trying to do is leverage the message trace logs for a dashboard that is pulling from two other log sources, Duo and Shibboleth, that have 'user' defined as their user field, which is my main input field.

Would it make sense to make an alias for src_user in the message trace logs to 'user' so my dashboard would work across all three log sources?

Thx

0 Karma

Ultra Champion

You can also resolve that in your dashboard query, by renaming the relevant fields in your search, to make them line up. How exactly to best do that depends on what exactly you want to achieve.

I'm not a big fan of putting very different data (sender and receiver) in 1 field, that is bound to lead to confusion. Especially since it is not compliant with the CIM Email data model.

0 Karma

Motivator

Thx for the feedback. I also am not a fan of changing the CIM email data model as well.

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!