Yup, you will lose your changes after upgrading Splunk. Easiest approach here would be to include a note in your internal documentation regarding what changes to test with the new version before upgrading your production splunk. For a fancy solution that's also upgrade-safe you'd fork your own sendemail.py into an app packaged with your own alert configuration that either overrides the existing configuration in the search app [see configuration file precedence in the docs] or defines an entirely new alert "send email with mad timeouts" to use for such cases. The latter approach has a big upside: regular users that define email alerts don't hurt themselves or your environment that badly because they're still protected by the standard timeouts. Finally to note, you can put in an ER with splunk support to make these timeouts configurable. ... View more