Splunk Search

How do I correlate email events when the key-id is not defined in all events?

Log_wrangler
Builder

So I have correlated email events before where there was a UID defined as a field for all transactions of a unique email session. For example, the event containing "subject" contained a UID=xyz123 and the event containing "sender" contained a UID=xyz123, and the event containing "recipient" contained a UID=xyz123, etc...

Now I am faced with transaction-events where the event containing "subject" has a UID=abc987, but the "to" and "from" events only have "abc987" and no field name...

I can regex the UID value out of the "to" and "from" events, but I have not been successful stitching the events together to create the complete email session...

For example, If I run this query below I can get the unique ids which will be contained in all events related to a unique email session... (by session I mean Subject, To, From, etc...)

index=mail sourcetype=mail | rex field=_raw "sendmail+\S+\s(?<stitcher>[[:alnum:]]+)"|fillnull value="null"| table stitcher | WHERE stitcher!="null" 

here are some sample results

stitcher
w9FD0v3f024155
w9CCWGaF023575
w9CCAwjU026498
w9AEM7sO030350
w9ADp31g031379
w993gkLc016485
w993gjU0016459
w993UuOr000878
w9CDhH42016767
w9CDV93a026891
w9CDVAv6018597

If I search with each of theses UIDs 1 by 1, I would only get the events related to a specific email.

So I tried a number of subsearches but I am having no luck...I need to feed the list back into a search and be able to list out the subject, to, and from, by UID...

Looking for advice.
Thank you

Tags (3)
0 Karma
1 Solution

jlelli
Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

View solution in original post

0 Karma

jlelli
Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

0 Karma

Log_wrangler
Builder

that is an idea I will try.... thank you

0 Karma

Log_wrangler
Builder

I had to use a combo of regex and transaction. Thx

0 Karma
Get Updates on the Splunk Community!

Upcoming Webinar: Unmasking Insider Threats with Slunk Enterprise Security’s UEBA

Join us on Wed, Dec 10. at 10AM PST / 1PM EST for a live webinar and demo with Splunk experts! Discover how ...

.conf25 technical session recap of Observability for Gen AI: Monitoring LLM ...

If you’re unfamiliar, .conf is Splunk’s premier event where the Splunk community, customers, partners, and ...

A Season of Skills: New Splunk Courses to Light Up Your Learning Journey

There’s something special about this time of year—maybe it’s the glow of the holidays, maybe it’s the ...