Tech Talk Recap   From Chaos to Control: Scaling Splunk Cloud with Infrastructure as Code Managing apps in Splunk Cloud Platform becomes complex quickly, as an average stack of about 50 applications introduces significant configuration, versioning, and operational overhead.  It can be challenging due to the number of applications involved. Even with around 50 apps, teams face considerable configuration, versioning, and operational tasks. Infrastructure as code (IaC) offers a solution by enabling teams to manage infrastructure through files rather than manual actions. This approach shifts management from browser clicks or one-off CLI commands to repeatable definitions that describe the desired end state, providing clearer control over changes in complex cloud environments.  This is particularly important for app management in Splunk, where apps, users, tokens, and indexes must remain aligned. With the right workflow, teams can define these resources once, apply them consistently, and prevent drift across environments.    Why Declarative Workflows Matter  Imperative tools specify how to perform actions, such as creating files or restarting services. Declarative tools specify the desired result, and the tool determines how to achieve that state.  The ACS CLI is mostly imperative, requiring explicit actions like restarting a stack. While creating apps or users is closer to declarative intent, the CLI lacks shared state, so independent changes by multiple users can cause inconsistencies.  Declarative workflows allow defining the target state once and comparing it with the current state, enabling safer and more predictable management of changes in Splunk Cloud.    What Infrastructure as Code Improves  Storing configuration in CI/CD workflows brings benefits such as:  Auditability: Changes stored as code (usually in Git) provide visibility into what changed, when, and in what order.  Compliance and Access Control: Essential in regulated environments (e.g., SOC2, HIPAA), supported by CI/CD platforms like GitHub Actions, GitLab CI, and Jenkins with branch and environment variable management.  Reproducibility and Rollback: Code-driven definitions allow consistent recreation of environments and easy rollback to previous configurations if issues arise.  These benefits depend on how Terraform manifests are written, emphasizing:  Separation of concerns through files and modules  Environmental parity across development, staging, and production  Reduced risk of storing secrets in source code by adopting safer patterns    Using Terraform for Splunk Cloud App Management  Terraform, developed by HashiCorp (now part of IBM), is a popular IaC solution using its own .tf manifest format. Teams define desired infrastructure and Terraform aligns actual infrastructure accordingly.  For Splunk Cloud Platform, the Terraform provider is available in the Terraform Registry under providers/Splunk/scp. The provider acts like a library with resources functioning as functions within it, supporting:  HEC tokens  Indexes  IP allow lists  Roles  Users  Splunkbase apps  Private apps  This provides a broad set of building blocks for managing Splunk Cloud through code.    Managing Splunk Cloud through code     Building a Practical Example with Splunkbase and GitHub A practical example uses the Splunk Add-on for GitHub (Splunk TA for GitHub) from Splunkbase. The goal is to install the app with Terraform, configure it, and use it to bring GitHub audit activity into Splunk Cloud. The initial manifest defines the Splunk provider, pins a provider version, and includes connection details for Splunk Cloud and Splunkbase. Hard-coded credentials in the first version are noted as a poor practice due to security risks. Improvements include using Terraform variables to separate sensitive values from the infrastructure blueprint, allowing loading from .tfvars files. The manifest defines both a Splunkbase app resource and an scp_user resource for a user accessing the app. Using references between resources reduces duplication and lets Terraform infer dependencies automatically. Practical Example with Splunkbase and GitHub     Practical Example with Splunkbase and GitHub   Reducing Repetition and Managing Dependencies Terraform allows referencing resource attributes directly to reduce repetition and help understand resource order. Defining locals stores reusable values like app names, simplifying configuration across resources. Explicit dependency declarations using depends_on ensure correct resource creation order, such as installing the app before creating the user, which is critical in Splunk Cloud. A distinction exists between Terraform and Splunk naming: The label in the resource line is Terraform’s internal name The name parameter appears in the Splunk UI Changing one does not necessarily change the other   From Manifest to Running Resources The Terraform workflow includes: Terraform init: Downloads required providers and sets up the working directory. Terraform plan: Compares current state with desired state, showing planned changes without applying them. Terraform apply: Creates or updates resources as defined. Existing resources are refreshed, and unnecessary changes are skipped, ensuring idempotency. This contrasts with imperative tooling, where repeated runs may recreate actions unnecessarily. Terraform’s declarative approach ensures no changes if the desired state is already met, ideal for CI/CD pipelines.   Extending the Workflow To GitHub Adding the GitHub provider and github_repository resource, Terraform creates a repository in a GitHub account. The Splunk Add-on for GitHub collects audit data from this repository into Splunk Cloud, closing the loop: Terraform installs and configures the Splunk app Terraform creates the GitHub resource The app collects GitHub activity Splunk displays the activity in search     Extending to Github       For more Splunk Cloud Application Management in Terraform go here.   Improving Security and Quality in Terraform Definitions Infrastructure as code enables automated analysis and guardrails. Tools like Checkov scan Terraform manifests for errors and misconfigurations, flagging weak password patterns and other issues. Snyk can scan Terraform definitions for cloud resources like AWS and GCP, though Splunk-specific Terraform definitions are not currently supported. Nonetheless, automated checks help catch issues earlier than manual UI or CLI management.   Splunk Cloud Application Management in Terraform     Terraform Definitions   Why Remote State Matters for Teams Local Terraform state files work for simple cases but pose challenges in team environments: Multiple administrators managing the same resources can cause conflicts without shared state. Local state loss (e.g., lost laptop) risks losing infrastructure state history, complicating recovery. Terraform supports remote state backends such as S3, Kubernetes, Postgres, Cassandra, and Terraform Cloud. Using Terraform Cloud provides a centralized, durable record of operations with details like: Who triggered the run Terraform version used Tags (e.g., staging) Full run history including successes and failures   Practical Takeaways for Splunk Teams For teams new to managing Splunk Cloud apps with Terraform: Start with declarative infrastructure basics Use the Splunk Cloud provider from the Terraform Registry Store sensitive values in variables and .tfvars files, not directly in manifests Reduce duplication with resource references and locals Use terraform plan before terraform apply Add remote state for team workflows Scan manifests with tools like Checkov and Snyk   Explore The Next Step Terraform offers a practical way to manage Splunk Cloud Platform resources with consistency, auditability, and control. The provider supports operations from app installation to managing users, indexes, and tokens, fitting naturally into CI/CD workflows. To learn more, explore the Splunk Cloud Terraform provider in the Terraform Registry, browse apps on Splunkbase, and follow the full GitHub-based example shared in the community. For additional technical sessions, join upcoming Splunk Tech Talks and engage right here on community.splunk.com.   Watch the full Tech Talk replay here   Watch the full story >       Get more out of Splunk with applications Read more about Splunk Cloud Application Management in Terraform     Behind the mic: Gregory Reshetniak, Senior Product Manager, Splunk.   Community Member: greshetniak I’m a technical product manager in the cloud / platform engineering domain. I bridge between teams engineering cutting edge solutions using the latest technology to business stakeholders and clients. Proponent of data driven decision making, I’m very active in strategic discussions, looking to perfect definitions, goals, metrics and missions. More than happy to crunch numbers to better understand the task at hand. I’ve been keenly interested in UX discipline since before such roles existed on a market - I enjoy building and testing prototypes, conducting user interviews, ideating new product features and studying analytics data for insights about the customer.   About Terraform Terraform is an open-source infrastructure as code software tool created by HashiCorp. It enables users to define and provision data center infrastructure using a declarative configuration language known as HashiCorp Configuration Language (HCL). Terraform manages infrastructure lifecycle through repeatable, version-controlled configuration files, supporting a wide range of providers and platforms. It helps teams automate infrastructure deployment, improve consistency, and reduce manual errors, making it a popular choice for cloud and on-premises infrastructure automation. ... View more