Forward ingested data

[javialborsan]

Hi there.

I'm just getting started with Splunk so I'm not familiar with it yet. 

We have this use case where we ingest data coming from Azure logs into Splunk, but we also need this data to be forwarded to AWS EventBridge or Lambda, for example.

For business simplicity reasons, we are wondering if Splunk can 'forward' the data it ingests into a third party, AWS in this case, so that our final ideal setup would be Azure sending data to Splunk, Splunk ingesting this data, Splunk sending this data to AWS, AWS ingesting this data. 

Is this even possible?

Thanks,

Javi 

[ddrillic]

Copilot said - 

Hi Javi,

Yes, it’s possible to forward data from Splunk to third-party services like AWS EventBridge or Lambda, but it’s not a native feature out of the box. Splunk primarily ingests and indexes data, but you can achieve this forwarding through a few approaches:

1. HTTP Event Collector (HEC) + AWS Integration
   Splunk can send data via HTTP Event Collector to an endpoint. You could set up an AWS API Gateway or Lambda function to receive these events and then route them to EventBridge.

2. Splunk Add-ons or Apps
   There are Splunkbase apps and add-ons for AWS that can help with integration. For example, the Splunk Add-on for AWS is typically used for ingestion, but custom scripts or modular inputs can be adapted for forwarding.

3. Custom Scripts / Modular Outputs
   Splunk supports custom alert actions or scripted outputs. You can configure Splunk to trigger a script whenever new data matches certain criteria, and that script can call AWS APIs to forward the data.

4. Third-Party Middleware
   Some organizations use a message bus (like Kafka or Kinesis) as an intermediary between Splunk and AWS services for scalability and reliability.

Summary:
Splunk doesn’t automatically forward all ingested data to AWS, but with HEC, custom alert actions, or middleware, you can build a pipeline where Azure → Splunk → AWS EventBridge/Lambda works as you described.

Would you like me to draft a step-by-step guide for setting up Splunk to forward data to AWS Lambda or EventBridge? Or should I prepare a diagram of the architecture for this flow?

[livehybrid]

Hi @ddrillic 

Its apparent that Copilot isnt very experienced with Splunk Cloud, Im not sure if any of those suggestions are even possible in Splunk Cloud, nevermind a recommended approach. 

Please remember that users use this community as a trusted source of information from other community users with experience in the issues they face, therefore LLM outputs really arent a good idea.

In relation to outputs, the Splunk Cloud Platform Service Details states:

does not support outputs of any kind at the indexer tier, including custom search commands, such as cefout (bundled with Splunk App for CEF). This helps ensure reliable and fault-tolerant performance of your Splunk Cloud Platform environment.

Whilst Splunk Cloud can receive HEC, it wouldnt be possible to configure it to send outputs apart from through supported Ingest Actions destinations (eg S3) unless Ingest Processor was implemented to sit infront of Splunk Cloud. 

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

Come on. Don't blindly copy-paste LLM's hallucinations.

You can't forward events to HEC. You can forward events to S2S over HTTP using httpout but only Splunk (and possibly one other solution which shall not be named here) supports receiving data this way.

Modular inputs are for input, as the name says. If they were to forward data outward you'd need to implement it manually. That's kinda pointless.

Forwarding via custom actions on alerts is not really a feasible solution. For small amounts of data it probably could be done but it's not a very intuitive and easy to maintain solution.

Third party middleware is completely off-topic. The question was whether Splunk on its own can forward data to other systems.

[livehybrid · Accepted solution]

Hi @javialborsan 

In Splunk Cloud the only way you could forward data ingested to Splunk would be using Ingest Actions or Ingest Processor (More complex setup) to save the ingested logs to Amazon S3 as JSONL files which you could then process using AWS Lambda? 

Alternatively you may be able to send from your Azure source to multiple locations if preferred? 

🌟 Did this answer help you? If so, please consider:

• Adding karma to show it was useful
• Marking it as the solution if it resolved your issue
• Commenting if you need any clarification

Your feedback encourages the volunteers in this community to continue contributing

[PickleRick]

Yes. Splunk can be made to forward data to several outputs of different types but depending on the type of the input it can be more or less tricky and might require additional components. It would be best if you discussed your specific use case with your local Splunk Partner which will help you find most fitting solution.