Splunk Search

How do I correlate email events when the key-id is not defined in all events?

Builder

So I have correlated email events before where there was a UID defined as a field for all transactions of a unique email session. For example, the event containing "subject" contained a UID=xyz123 and the event containing "sender" contained a UID=xyz123, and the event containing "recipient" contained a UID=xyz123, etc...

Now I am faced with transaction-events where the event containing "subject" has a UID=abc987, but the "to" and "from" events only have "abc987" and no field name...

I can regex the UID value out of the "to" and "from" events, but I have not been successful stitching the events together to create the complete email session...

For example, If I run this query below I can get the unique ids which will be contained in all events related to a unique email session... (by session I mean Subject, To, From, etc...)

index=mail sourcetype=mail | rex field=_raw "sendmail+\S+\s(?<stitcher>[[:alnum:]]+)"|fillnull value="null"| table stitcher | WHERE stitcher!="null" 

here are some sample results

stitcher
w9FD0v3f024155
w9CCWGaF023575
w9CCAwjU026498
w9AEM7sO030350
w9ADp31g031379
w993gkLc016485
w993gjU0016459
w993UuOr000878
w9CDhH42016767
w9CDV93a026891
w9CDVAv6018597

If I search with each of theses UIDs 1 by 1, I would only get the events related to a specific email.

So I tried a number of subsearches but I am having no luck...I need to feed the list back into a search and be able to list out the subject, to, and from, by UID...

Looking for advice.
Thank you

Tags (3)
0 Karma
1 Solution

Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

View solution in original post

0 Karma

Path Finder

If you have common unique values for different fields, would not be easier to rename them all and then apply a transaction command on the renamed field? Something like:

| rename subject as UID | rex "sendmail+\S+\s(?<UID>[[:alnum:]]+)" | transaction UID 

If the time and format of the events are always the same you can add further options to the transaction, something like
| transaction UID startswith=subject endswith=to and/or | transaction UID maxevents=3

View solution in original post

0 Karma

Builder

that is an idea I will try.... thank you

0 Karma

Builder

I had to use a combo of regex and transaction. Thx

0 Karma
State of Splunk Careers

Access the Splunk Careers Report to see real data that shows how Splunk mastery increases your value and job satisfaction.

Find out what your skills are worth!