Splunk TA for Symantec Brightmail

[laklubinsplunk]

Anyone have TA for Symantec brightmail.

[tomasmoser]

I am working on brand new Symantec Messaging Gateway (Brightmail) TA right now.  Once its done I will share it! This one si more comprehensive than the one currenlty available on Splunkbase. 

[sandroherman]

Hi everybody!

We are using the following field extraction, in compliance with CIM(1):

^\<\d+\>(?:.+\d+:\d+:\d+)\s+(?<dvc>\w+)\s+(?<process>[a-z]+)\[(?<process_number>\d+)\]:\s+(?<process_id>[^\|]+)\|(?<internal_message_id>[^\|]+)\|(?<message_info>\w+[^\|])?\|?(?<x1>[^\|]+)?\|?(?<x2>[^\|]+)?\|?(?<x3>[^\|]+)?\|?(?<xn>[^$|\s]+.*)?$

We define the fields like ´{field}=value´ and we always use subsearch to find something :

sourcetype=smg IRCPTACTION

       [search sourcetype=smg *gmail.com | stats count by internal_message_id| table internal_message_id]

| eval {message_info}=x1, audit_id=internal_message_id
| transaction audit_id maxpause=15min

We tried another regex, but it doesn't have all fields like SPF, DKIM and DMARC.

1 https://docs.splunk.com/Documentation/CIM/4.15.0/User/Email
2 http://alec.dhuse.com/wp/2016/09/

[to4kawa]

thanks @sandroherman
I haven't know the audit_id of SMG is the internal_message_id.

but there is many xn, transformes.conf field extraction is better, I guess.

Making summary index by report, this is best practice.

[sandroherman]

Hi. Did you create the summary index? which query did you use

[to4kawa]

hi @sandroherman
I haven't done it yet.

....
| stats values(SENDER) as from values(RECIPIENT) as to values(SUBJECT) as subject values(FIRED) as fired by audit_id
| eval to=mvjoin(to,";"), fired=mvjoin(split(fired,"|"),"; ")
| collect smg_index

There are more fields, this is for example.

Do you have sample logs?
I'll extract fields and make the query.

[sandroherman]

Look this link:
https://regex101.com/r/kR0iS8/1

Do you prefer stats against transaction? And about events out of window time?

are you aware of this information?
"All events in a summary index have stash as their default source type. If you use a command like collect to change their source type to anything other than stash, you will incur license usage charges for those events".

[to4kawa]

thanks @sandroherman

I see license issue.

for example:
1. create summary index

....
| stats min(_time) as _time value(*) as * by message_id
| eval summary_name="SMG_index"
| collect

1. search summary_index

   index=stash summary_name=SMG_index "you want"

How's this?

and transaction is too slow. SMG audit_id(message_id in your REGEX) is unique.
stats is better.

[to4kawa · Accepted solution]

manual regex
verdict message
UNTESTED
splunk-field-extractions-for-symantec-messaging-gateway-a-k-a-brightmail-syslogs
Log format of message audit logs for remote syslog

previous answers makes REGEX.
I collect some relative links.
I don't know TA. Please tell me if you find.

|makeresults
| eval _raw="14:45 Symantec_Brightmail <142>Jul 3 14:51:36 mailrelay ecelerity: 1341316296|c0a88701-b7cedae000003dec-a7-4ff2dcc83a30|ACCEPT|192.168.115.130:51998
14:45 Symantec_Brightmail <142>Jul 3 14:51:14 mailrelay bmserver: 1341316274|c0a88701-b7cedae000003dec-91-4ff2dcb2aaaf|VERDICT|xxx123@gmail.com|senderauth_batv_sign|default|static bounce attack prevention sign
14:45 Symantec_Brightmail <142>Jul 3 14:51:10 mailrelay bmserver: 1341316270|c0a88701-b7cedae000003dec-8c-4ff2dcae65dc|VERDICT|mir@mac.com|senderauth_batv_sign|default|static bounce attack prevention sign
14:45 Symantec_Brightmail <142>Jul 3 14:51:15 mailrelay ecelerity: 1341316275|c0a88701-b7cedae000003dec-92-4ff2dcb3dfaa|ACCEPT|192.168.115.132:51723
14:45 Symantec_Brightmail <142>Jul 3 14:51:05 mailrelay ecelerity: 1341316265|c0a88701-b7cedae000003dec-86-4ff2dca8f358|DELIVER|212.199.239.178:25|edi@perry5y.co.il
14:44 Symantec_Brightmail <142>Jul 3 14:50:53 mailrelay ecelerity: 1341316221|c0a88701-b7cedae000003dec-52-4ff2dc7c9c9d|SENDER|shlomy1006+caf_=sshahar=xyx.il@gmail.com
14:44 Symantec_Brightmail <142>Jul 3 14:50:44 mailrelay bmserver: 1341316244|c0a88701-b7cedae000003dec-71-4ff2dc941242|VERDICT|m32@wanna.com|senderauth_batv_sign|default|static bounce attack prevention sign
14:45 Symantec_Brightmail <142>Jul 3 14:51:14 mailrelay bmserver: 1341316274|c0a88701-b7cedae000003dec-91-4ff2dcb2aaaf|VERDICT|rgakanov@gmail.com|senderauth_batv_sign|default|static bounce attack prevention sign"
| makemv delim="
" _raw
| stats count by _raw
| eval _raw=replace(_raw,".*>","")
| rename COMMENT as "this is sample, https://www.symantec.com/connect/forums/format-smg-log-output"
| rex "(?<timeStamp>^.+) mailrelay (?<mta>\S+): (?<sessionId>\d+)\|(?<auditId>.*?)\|(?<msg>.*)"

this is sample,
For bmserver log, basically this format.
so, extract msg to as_you_like.
Of course the connection log is separate, so it needs to extract the fields with it.

Want to make a TA?
I'll help you

[laklubinsplunk]

I haven't find any TA or Regex posted in community.

[laklubinsplunk]

@to4kawa do you the TA ?

[to4kawa]

I don't have TA and logs.
but If there is logs, we can extract fields.
mail is sensitive, these must sanitize.