topic How do I correlate email events when the key-id is not defined in all events? in Splunk Search

How do I correlate email events when the key-id is not defined in all events?

Log_wrangler — Mon, 15 Oct 2018 23:23:48 GMT

So I have correlated email events before where there was a UID defined as a field for all transactions of a unique email session. For example, the event containing "subject" contained a UID=xyz123 and the event containing "sender" contained a UID=xyz123, and the event containing "recipient" contained a UID=xyz123, etc...

Now I am faced with transaction-events where the event containing "subject" has a UID=abc987, but the "to" and "from" events only have "abc987" and no field name...

I can regex the UID value out of the "to" and "from" events, but I have not been successful stitching the events together to create the complete email session...

For example, If I run this query below I can get the unique ids which will be contained in all events related to a unique email session... (by session I mean Subject, To, From, etc...)

index=mail sourcetype=mail | rex field=_raw "sendmail+\S+\s(?<stitcher>[[:alnum:]]+)"|fillnull value="null"| table stitcher | WHERE stitcher!="null"

here are some sample results

stitcher
w9FD0v3f024155
w9CCWGaF023575
w9CCAwjU026498
w9AEM7sO030350
w9ADp31g031379
w993gkLc016485
w993gjU0016459
w993UuOr000878
w9CDhH42016767
w9CDV93a026891
w9CDVAv6018597

If I search with each of theses UIDs 1 by 1, I would only get the events related to a specific email.

So I tried a number of subsearches but I am having no luck...I need to feed the list back into a search and be able to list out the subject, to, and from, by UID...

Looking for advice.
Thank you

Re: How do I correlate email events when the key-id is not defined in all events?

jlelli — Tue, 16 Oct 2018 07:30:20 GMT

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

Re: How do I correlate email events when the key-id is not defined in all events?

Log_wrangler — Tue, 16 Oct 2018 19:39:37 GMT

that is an idea I will try.... thank you

Re: How do I correlate email events when the key-id is not defined in all events?

Log_wrangler — Fri, 09 Nov 2018 16:37:47 GMT

I had to use a combo of regex and transaction. Thx