I would like to understand if the following requirement can be made to work..
We are ingesting AWS Cloudtrail events, via the Splunk Add on for AWS. The app is installed on our indexer so we aren't currently going via a heavy forwarder. The Cloudtrail event data currently goes from our AWS account to an S3 bucket and then into Splunk via SQS.
Our client has a new requirement to ensure that only particular Cloudtrail events, like 'password reset' or 'failed login' are sent to a third party log rhythm server but are also still indexed in our Splunk environment. I've read the documentation and this seems to fall under the following articles:
https://docs.splunk.com/Documentation/Splunk/8.0.1/Search/Forwarddatatothirdpartysystems
https://docs.splunk.com/Documentation/Splunk/latest/Forwarding/Routeandfilterdatad#Replicate_a_subse...
I've tested with the following configuration, just searching for a simple error string. However I end up with data indexed TWICE at our end.
props.conf
[default]
TRANSFORMS-routing=TestErrors
Transforms.conf
[TestErrors]
REGEX = awerror12345
DEST_KEY = _MetaData:Index
FORMAT = index
DEST_KEY=_TCP_ROUTING
FORMAT=TestErrors
outputs.conf
[tcpout:TestErrors]
server=x.x.x.x:9999
Is it possible to filter events using regex, send them to a third party AND index them at our end possible?
