Hello Splunkers,
We are receiving config notifications, CloudTrail and others from AWS through Kinesis - the general pattern is:
Config -> Event Rules -> Event Hub -> Kinesis -> HEC (indexer cluster)
This works, what seems flawlessly.
We separate the data into indexes based on the account number due to security policies so I've set up props/transforms to do this.
This seems to work most of the time and 77% of the traffic ends up in the correct index but there is 33% (on avg) that ends up in the default index of the HEC (aws_config)
Is it possible that the transforms aren't triggering all the time?
The events are identical format; sourcetype and source are identical.
Here is the transforms:
[aws-account1]
REGEX = 010016492034
DEST_KEY = _MetaData:Index
FORMAT = aws-account1
Props:
[aws:config:notification]
TRANSFORMS-aws_config_notification=aws-account1
Am I missing something here? Is there anything I should look for in internal?
I remember years ago in training an example where props hierarchy would mess with data when there was multiple props/transforms, and the intermittent nature *might* make sense but I have no idea where to troubleshoot this.
The HEC is on a cluster of indexers so the config is all via the CM, thus no differences.
Any suggestions would be greatly appreciated!
Cheers!
