Not entirely using Splunk...
For each alert, you could specify a script to run, in addition to other actions that you wish to take.
In the script, write all of the script parameters plus a timestamp to a file, which you then use as an input to Splunk. You could either use the file as a lookup table or index it like a monitor input.
If you use the file as a lookup table, you could also update it with a status for the alert, such as "resolved". Of course, you would have to write a script or program to do this update.
Other References:
Scripted Alerts in documentation
Use Splunk Alerts to create a ticket in your ticketing system
