Is the smtp server configured in your splunk instance? And you don't need to configure the alert to see if it works or not. Put a sendmail command to just test it out from search itself
_http://docs.splunk.com/Documentation/Splunk/6.0.1/SearchReference/Sendemail
