Sizing my Splunk for MSExchange App

[maverick]

I would like to know what to expect with regard to Splunk's daily indexing volume for my Splunk for MSExchange App.

The deployment guide for this app mentions that it indexes the following:

File inputs:
• IIS logs for the Exchange server roles running on IIS
• pop3 and IMAP transport logs
• Windows Event logs
  - Exchange audit logs
  - Application logs, such as Forefront security logs

Scripted inputs:
• Performance monitoring data on all Mailbox Store servers
• Senderbase/reputation data. (This feature needs internet access to function, as it looks up the
reputation score for your email users.)

What daily capacity is expected per average MSExchange server and is is different on 2003 vs 2007?

Also, how might this estimate change if I add ten or hundred more servers? Is it linear in scale, etc?

[ahall_splunk · Accepted solution]

Your SE has a sizing sheet available to them if you know nothing about your Exchange sizing (which will be unusual). There are marginal differences between Exchange 2007 and 2010, but we will take them into account. Your sizing is broken into five "bits"

1. The size of the message tracking logs
2. The size of the IIS logs on the client access servers
3. The size of the POP3 and IMAP4 transport logs
4. The perfmon counters being recorded
5. The Powershell scripts being recorded

You can get the message tracking logs and IIS logs by keeping them for a few days and computing the size/day. I don't recommend running the POP3 and IMAP4 transport logs - their size far out-weighs their usefulness. Powershell is miniscule in the grand scheme of things - just add about 10% to the overall number you get and you will probably have handled it. That leaves perfmon. The average perfmon event is about 200 bytes long. Use the following numbers:

• 174 perfmon events/minute on a Client Access Server
• 156 perfmon events/minute on a Hub or Edge Transport Server
• 230 perfmon events/minute on a Mailbox Store (count cluster members individually)

From this, you can calculate the amount of perfmon data approximately coming in. Add up all the bits and you have an estimate of the expected additional indexing your splunk instance will do when you start doing Splunk App for Microsoft Exchange.

[cramasta]

How many active users per day do you have in your exchange environment and how often are they connecting? What protocol are the majority of users connecting with?