Example of how to measure continuous delivery test success rates?

[sloshburch]

Does anyone have examples of how to use Splunk to measure continuous delivery test success rates?

[sloshburch · Accepted solution]

The Splunk Product Best Practices team helped produce this response. Read more about example use cases in the Splunk Platform Use Cases manual.

For more information about this example see KPIs and thresholds for the ITSI Module for Continuous Delivery in the Splunk ITSI Module for Continuous Delivery manual.

Application development teams can find it overwhelming to deal with a flurry of production changes inherent in a continuous integration and continuous delivery (CI/CD) model. Track these changes to maintain security compliance and have an audit trail for the life cycle of a host or application. Over time, you can use that data to demonstrate trends to improve your teams' performance.

Set up this example use case to measure continuous delivery test success rate.

Load data

How to implement: This example use case depends on event data from application performance monitoring platforms.

Install the appropriate add-ons for the platforms that exist in your environment.

• Splunk Add-on for Bamboo
• Splunk Add-on for JIRA
• Splunk App for Jenkins
• Splunk Plugin for Jenkins

The Splunk Plugin for Jenkins used in this post is not supported by Splunk, but is available for download from the Jenkins website as an open-source tool. You can find installation and configuration instructions in the Details tab of each Splunkbase item. Additional configuration details are available in the Configure the ITSI Module for Continuous Delivery in the Install and Configure the Splunk ITSI Module for Continuous Delivery manual.

Best practice: For more granular results with scripted inputs, you can increase the frequency at which the input runs using the interval setting in inputs.conf. Running the input more frequently consumes more storage, and running it less frequently uses less, which can affect license consumption. The default interval is 60 seconds. See Scripted Input in the input.conf topic of the Splunk Enterprise Admin manual.

Best practice: For all of the data inputs, specify a desired target index to provide a more sustainable practice for data access controls and retention models. By default, Splunk collects the data in the default index named main.

Get insights

Use the Splunk platform to see the percentage of successful tests.

Run the following search.

index=* tag=continuous_delivery tag=test_system

| mvexpand lifecycle_id 

| eval success_rate=if(status=="Successful" OR status="SUCCESS", 100, 0)

| timechart span=5min avg(success_rate)

Best practice: In searches, replace the asterisk in index=* with the name of the index that contains the data. By default, Splunk stores data in the main index. Therefore, index=* becomes index=main. Use the OR operator to specify one or multiple indexes to search. For example, index=main OR index=security. See About managing indexes and How indexing works in Splunk docs for details.

How to respond: Track changes in your environment to complete the following tasks as needed:

• Maintain security compliance and allow for a detailed lifecycle of a host or application
• Tie change requests to different departments
• Configure the system to send alerts when changes fail, are outside the normal window, and respond to unauthorized changes
• Address issues from your multi-stage build plans and configure the system to send notifications, alerts, and create tickets to resolve issues in real-time

Help

The Troubleshoot the Splunk ITSI Module for Continuous Delivery section in the Install and Configure the Splunk ITSI Module for Continuous Delivery manual lists troubleshooting resources you can apply to this example use case.

If no results appear, it may be because the add-ons were not deployed to the search heads, so the needed tags and fields are not defined. Deploy the add-ons to the search heads to access the needed tags and fields. See About installing Splunk add-ons in the Splunk Add-ons manual.

For troubleshooting tips that you can apply to all add-ons, see Troubleshoot add-ons in the Splunk Add-ons manual.

For more support, post a question to the Splunk Answers community.