I used it to analyze logs after test runs but there are APIs that you can use to interconnect your systems so that you could, for example, run a test, send a search to Splunk, get the results of the search from Splunk, examine the results, and take automatic action based on them. Splunk has been very active lately in adding new APIs and SDKs for this kind of thing. Probably the most popular is the REST API: http://dev.splunk.com/view/rest-api-tutorials/SP-CAAADQ6
In my experience, Splunk is most excellently suited for analyzing/summarizing/alerting on the output/logging of test runs. I used to build complex awk scripts for this in a performance lab but after I found Splunk, Splunk was all that I ever used.
