How to extract fields from email header to get email delivery time between each email server...

[melonman]

Hi,

I am trying to create email performance monitor using imap app. Using email header, I would like to get how long it takes to deliver email between each mail server, possibly using bar graph. This will give email service provider brief service assurance view.

I can get email using imap app. Next step is to extract timestamp that shows email arrival time at each mail server. (information after Received = )

The following is the email address we used for test.

Date = "13-jan-2011 15:10:28 +0900"
Return-Path = "<testuser1@mydomain.poc>"
Received = "from localhost by mail3.mydomain.poc with LMTP
    for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:28 +0900"
Received = "from mail3.mydomain.poc with LMTP by mail3.mydomain.poc (3.1.0/sieved-3-1-SW-build-1314)
    for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:28 +0900"
Received = "from mail2.mydomain.poc ([192.168.30.112])
    by mail3.mydomain.poc (Switch-3.3.3/Switch-3.3.3) with ESMTP id p0D6AR0U023082
    (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO)
    for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:28 +0900"
Received = "from mail1.mydomain.poc ([192.168.30.111])
    by mail2.mydomain.poc (Switch-3.3.3/Switch-3.3.3) with ESMTP id p0D6AQ1N023608
    for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:27 +0900"
Received = "from mail1.mydomain.poc (localhost.localdomain [127.0.0.1])
    by mail1.mydomain.poc (Switch-3.3.3/Switch-3.3.3) with ESMTP id p0D6APqB026204
    for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:26 +0900"
Received = "(from root@localhost)
    by mail1.mydomain.poc (Switch-3.3.3/Switch-3.3.0/Submit) id p0D6AOoj026203
    for testuser2@mydomain.poc; Thu, 13 Jan 2011 15:10:24 +0900"
From = "testuser1@mydomain.poc"
Message-Id = "<201101130610.p0D6AOoj026203@mail1.mydomain.poc>"
To = "testuser2@mydomain.poc"
Subject = "Mail Performance Check"
mailbox = "INBOX"
size = 1375
____________________  Message Body  ____________________
sentAt = 2011/01/13 15:10:24

There are multiple Received information, I need to get each arraival timestamp and possibly get the similar result to the following.

messageid                                             sentAt              mail_server time_taken ReceivedAt
----------------------------------------------------- ------------------- ----------- ---------- -------------------
201101130610.p0D6AOoj026203@mail1.mydomain.poc>"      2011/01/13 15:10:24 mail1       2          2011/01/13 15:10:28
                                                                          mail2       1
                                                                          mail3       1
201101130610.p0D6AOoj026203@mail1.mydomain.poc>"      2011/01/13 15:10:34 mail1       2          2011/01/13 15:10:38
                                                                          mail2       1
                                                                          mail3       1
201101130610.p0D6AOoj026203@mail1.mydomain.poc>"      2011/01/13 15:10:44 mail1       2          2011/01/13 15:10:48
                                                                          mail2       1
                                                                          mail3       1
201101130610.p0D6AOoj026203@mail1.mydomain.poc>"      2011/01/13 15:10:54 mail1       2          2011/01/13 15:10:58
                                                                          mail2       1
                                                                          mail3       1

Thanks!

--- added Jan 14, 2011

I can get the Received, but only the last Received value is shown. configuration includes MV_ADD=true.

# splunk search 'sourcetype="imap" mailbox="INBOX" Message_Id="*201101130610.p0D6AOoj026203*" From="testuser1@mydomain.poc" To="testuser2@mydomain.poc" | stats list(Received)' -auth admin:changeme

                                              list(Received)
-----------------------------------------------------------------------------------------------------------
from localhost by mail3.mydomain.poc with LMTP 
        for <testuser2@mydomain.poc>; Thu, 13 Jan 2011 15:10:28 +0900

How do I get multiple Received values?

[rene_niedermeye]

Same for me.
Anyone made progress with this?

[MartinMcNutt]

Was anymore progress made on this? I am looking to do something similar in our environment.

[fk319 · Accepted solution]

It looks like it can be configured in transfomrs.conf
http://www.splunk.com/base/Documentation/4.1.5/Admin/Transformsconf
Things of note are

MV_ADD = True

and

REGEX = (\S+) = (.*)

FORMAT = $1::$2

you still need to deal with multi-line and extract headers

A finial option is to build your own external program.

I know this is not of much help, but can see the need for parsing e-mail headers for my self down the road and would like to wish you luck.

[melonman]

Thanks, I tried with that configuration and the result is shown at the and of the question above. ( I added the information) I would like to know how to get multiple repeated values.