I am looking for a solution for my current environment:
- Data residing on AWS S3. This data is from various sources and we collect them to AWS S3 buckets
- We are planning to install HF under the same AWS account where the data is available on S3. This data should be injected from S3 to Heavy Forwarder (HF) and then from HF, it should get ingested into Indexer cluster
- Since we are getting the data from various different sources, do we need to install individual Splunk apps or add-ons for these data types on HF. Data may be Cylance, FireEye etc. data? Since couple of these apps require data ingestion directly from the source device, it seems we cannot use them for our purpose.
My question is: Should we directly inject data from S3 to HF and then from HF to Indexer cluster?
Here is a flow to show end to end picture:
AWS S3 (Data from sources) ->> AWS SQS ->> HF (with Splunk App for AWS to pull data from AWS SQS) ->> Indexer cluster
Thanks.
