Getting Data In

Possible to extract same value from different fields in props.conf?

jwalzerpitt
Influencer

For Exchange message trace logs I am extracting the user as following in the props.conf file:

EXTRACT-user = "RecipientAddress":"(?<user>\S+)@

I would also like to extract the user from the SenderAddress as follows:

EXTRACT-user = "SenderAddress":"(?<user>\S+)@

Can I extract from two different fields for the user field, or will this potentially cause confusion?

Thx

0 Karma
1 Solution

woodcock
Esteemed Legend

I would do it like this:

EXTRACT-user = "RecipientAddress":"(?<user_recipient>\S+)@
EXTRACT-user = "SenderAddress":"(?<user_sender>\S+)@
EVAL-user = mvappend(user_recipient, user_sender)

View solution in original post

0 Karma

woodcock
Esteemed Legend

I would do it like this:

EXTRACT-user = "RecipientAddress":"(?<user_recipient>\S+)@
EXTRACT-user = "SenderAddress":"(?<user_sender>\S+)@
EVAL-user = mvappend(user_recipient, user_sender)
0 Karma

jwalzerpitt
Influencer

Thx for the extracts.

My dashboard is currently set up to use 'user' as the value as my Shib and Duo logs have the user field. My message trace logs does not have the 'user' field, but 'src_user' instead. If I set the extracts you graciously provided, will this nullify the Email CIM data model of using 'src_user'?

Thx

0 Karma

woodcock
Esteemed Legend

You are free to change EVAL-user to EVAL-src_user or EVAL-some_other_field_name and it will work that way. If you keep it as-is, it will override any existing value of user for this sourcetype.

If you would like to prefer any existing value, then use this:

 EVAL-user = coalesce(user, mvappend(user_recipient, user_sender))

If you would like to keep both any existing value and this new value, then use this:

EVAL-user = mvappend(user, user_recipient, user_sender)

This covers all possible desires. Be sure to click Accept to close the question.

0 Karma

jwalzerpitt
Influencer

Thx for the various breakdowns

0 Karma

FrankVl
Ultra Champion

It might make more sense to put one in src_user and the other in user. Or better yet, have a look at the definition for the CIM Email data model: https://docs.splunk.com/Documentation/CIM/4.13.0/User/Email and adhere to that.

If you really want to generate a multi-valued field user, with both sender and recipient(s), extract each into a separate field (e.g. s_user and r_user) and then do something like:

EVAL-user = mvappend(s_user,r_user)

Or use a REPORT extraction like this:

props.conf

REPORT-user = exchange-user

transforms.conf

[exchange-user]
REGEX = "(?:RecipientAddress|SenderAddress)":"(?<user>\S+)@
MV_ADD = true

jwalzerpitt
Influencer

Frank,

Thx for the reply and info.

What I'm trying to do is leverage the message trace logs for a dashboard that is pulling from two other log sources, Duo and Shibboleth, that have 'user' defined as their user field, which is my main input field.

Would it make sense to make an alias for src_user in the message trace logs to 'user' so my dashboard would work across all three log sources?

Thx

0 Karma

FrankVl
Ultra Champion

You can also resolve that in your dashboard query, by renaming the relevant fields in your search, to make them line up. How exactly to best do that depends on what exactly you want to achieve.

I'm not a big fan of putting very different data (sender and receiver) in 1 field, that is bound to lead to confusion. Especially since it is not compliant with the CIM Email data model.

0 Karma

jwalzerpitt
Influencer

Thx for the feedback. I also am not a fan of changing the CIM email data model as well.

0 Karma
Get Updates on the Splunk Community!

Index This | I am a number, but when you add ‘G’ to me, I go away. What number am I?

March 2024 Edition Hayyy Splunk Education Enthusiasts and the Eternally Curious!  We’re back with another ...

What’s New in Splunk App for PCI Compliance 5.3.1?

The Splunk App for PCI Compliance allows customers to extend the power of their existing Splunk solution with ...

Extending Observability Content to Splunk Cloud

Register to join us !   In this Extending Observability Content to Splunk Cloud Tech Talk, you'll see how to ...