The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.
The best practice solution is to leverage Splunk AWS Project Trumpet to automate how you collect data for many of these popular data sources in your AWS account.
Splunk AWS Project Trumpet is an open-source tool provided by Splunk that allows you to select the data sources you want to collect, then specify the HEC token where Amazon Kinesis Data Firehose (KDF) should send the events to. Trumpet then deploys a CloudFormation template to create the appropriate AWS resources to begin streaming the events to Splunk HEC.
Although Splunk AWS Project Trumpet leverages Splunk-supported solutions such as the Amazon Kinesis Data Firehose to Splunk integration, it is not a Splunk-supported solution.
You can read more about Splunk AWS Project Trumpet in the blog Automating AWS Data Ingestion into Splunk on Splunk Blogs. You can find the utility itself and additional details at splunk-aws-project-trumpet on Github.
Here are some recommendations for how to implement Splunk AWS Project Trumpet:
- If you are a Splunk Cloud customer, first file a ticket with Splunk Support to enable HEC for use with Kinesis Firehose. Splunk Cloud operations will provide you with a destination URL to send your KDF data.
- If you are a Splunk customer running Splunk Enterprise on-prem, we recommend that you deploy a heavy forwarder in your AWS environment with HEC enabled. This enables KDF to forward the data to the heavy forwarder first, then to Splunk Enterprise as a data-collection component.
- If you are running the Splunk platform in your own AWS environment, deploy a HEC token to all your indexers, and place a classic load balancer in front of the indexers to distribute the KDF traffic across all indexers.
Add-on requirements:
- Deploy the Splunk Add-on for AWS on your search heads, indexers, and heavy forwarder (if applicable).
The following video provides a visual walk through of deploying Splunk AWS Project Trumpet in your own environment.
The Splunk Product Best Practices team provided this response. Read more about How Crowdsourcing is Shaping the Future of Splunk Best Practices.
The best practice solution is to leverage Splunk AWS Project Trumpet to automate how you collect data for many of these popular data sources in your AWS account.
Splunk AWS Project Trumpet is an open-source tool provided by Splunk that allows you to select the data sources you want to collect, then specify the HEC token where Amazon Kinesis Data Firehose (KDF) should send the events to. Trumpet then deploys a CloudFormation template to create the appropriate AWS resources to begin streaming the events to Splunk HEC.
Although Splunk AWS Project Trumpet leverages Splunk-supported solutions such as the Amazon Kinesis Data Firehose to Splunk integration, it is not a Splunk-supported solution.
You can read more about Splunk AWS Project Trumpet in the blog Automating AWS Data Ingestion into Splunk on Splunk Blogs. You can find the utility itself and additional details at splunk-aws-project-trumpet on Github.
Here are some recommendations for how to implement Splunk AWS Project Trumpet:
- If you are a Splunk Cloud customer, first file a ticket with Splunk Support to enable HEC for use with Kinesis Firehose. Splunk Cloud operations will provide you with a destination URL to send your KDF data.
- If you are a Splunk customer running Splunk Enterprise on-prem, we recommend that you deploy a heavy forwarder in your AWS environment with HEC enabled. This enables KDF to forward the data to the heavy forwarder first, then to Splunk Enterprise as a data-collection component.
- If you are running the Splunk platform in your own AWS environment, deploy a HEC token to all your indexers, and place a classic load balancer in front of the indexers to distribute the KDF traffic across all indexers.
Add-on requirements:
- Deploy the Splunk Add-on for AWS on your search heads, indexers, and heavy forwarder (if applicable).
The following video provides a visual walk through of deploying Splunk AWS Project Trumpet in your own environment.
