Getting Data In

Sizing my Splunk for MSExchange App

maverick
Splunk Employee
Splunk Employee

I would like to know what to expect with regard to Splunk's daily indexing volume for my Splunk for MSExchange App.

The deployment guide for this app mentions that it indexes the following:

File inputs:
• IIS logs for the Exchange server roles running on IIS
• pop3 and IMAP transport logs
• Windows Event logs
  - Exchange audit logs
  - Application logs, such as Forefront security logs

Scripted inputs:
• Performance monitoring data on all Mailbox Store servers
• Senderbase/reputation data. (This feature needs internet access to function, as it looks up the
reputation score for your email users.)

What daily capacity is expected per average MSExchange server and is is different on 2003 vs 2007?

Also, how might this estimate change if I add ten or hundred more servers? Is it linear in scale, etc?

1 Solution

ahall_splunk
Splunk Employee
Splunk Employee

Your SE has a sizing sheet available to them if you know nothing about your Exchange sizing (which will be unusual). There are marginal differences between Exchange 2007 and 2010, but we will take them into account. Your sizing is broken into five "bits"

  1. The size of the message tracking logs
  2. The size of the IIS logs on the client access servers
  3. The size of the POP3 and IMAP4 transport logs
  4. The perfmon counters being recorded
  5. The Powershell scripts being recorded

You can get the message tracking logs and IIS logs by keeping them for a few days and computing the size/day. I don't recommend running the POP3 and IMAP4 transport logs - their size far out-weighs their usefulness. Powershell is miniscule in the grand scheme of things - just add about 10% to the overall number you get and you will probably have handled it. That leaves perfmon. The average perfmon event is about 200 bytes long. Use the following numbers:

  • 174 perfmon events/minute on a Client Access Server
  • 156 perfmon events/minute on a Hub or Edge Transport Server
  • 230 perfmon events/minute on a Mailbox Store (count cluster members individually)

From this, you can calculate the amount of perfmon data approximately coming in. Add up all the bits and you have an estimate of the expected additional indexing your splunk instance will do when you start doing Splunk App for Microsoft Exchange.

View solution in original post

ahall_splunk
Splunk Employee
Splunk Employee

Your SE has a sizing sheet available to them if you know nothing about your Exchange sizing (which will be unusual). There are marginal differences between Exchange 2007 and 2010, but we will take them into account. Your sizing is broken into five "bits"

  1. The size of the message tracking logs
  2. The size of the IIS logs on the client access servers
  3. The size of the POP3 and IMAP4 transport logs
  4. The perfmon counters being recorded
  5. The Powershell scripts being recorded

You can get the message tracking logs and IIS logs by keeping them for a few days and computing the size/day. I don't recommend running the POP3 and IMAP4 transport logs - their size far out-weighs their usefulness. Powershell is miniscule in the grand scheme of things - just add about 10% to the overall number you get and you will probably have handled it. That leaves perfmon. The average perfmon event is about 200 bytes long. Use the following numbers:

  • 174 perfmon events/minute on a Client Access Server
  • 156 perfmon events/minute on a Hub or Edge Transport Server
  • 230 perfmon events/minute on a Mailbox Store (count cluster members individually)

From this, you can calculate the amount of perfmon data approximately coming in. Add up all the bits and you have an estimate of the expected additional indexing your splunk instance will do when you start doing Splunk App for Microsoft Exchange.

cramasta
Builder

How many active users per day do you have in your exchange environment and how often are they connecting? What protocol are the majority of users connecting with?

0 Karma
Get Updates on the Splunk Community!

The Splunk Success Framework: Your Guide to Successful Splunk Implementations

Splunk Lantern is a customer success center that provides advice from Splunk experts on valuable data ...

Splunk Training for All: Meet Aspiring Cybersecurity Analyst, Marc Alicea

Splunk Education believes in the value of training and certification in today’s rapidly-changing data-driven ...

Investigate Security and Threat Detection with VirusTotal and Splunk Integration

As security threats and their complexities surge, security analysts deal with increased challenges and ...