AWS Logs via Lamda to HF drop preceding key from fieldname

[cybersecnutant]

Hello, I've got a Lamda function exporting AWS logs via HEC to my HF's to my indexers.

Unfortunately, the AWS logs are coming in with event.* as all of the field names, whereas the Splunk_TA_aws addon is expecting *.

I can easily do a rename event.* as a *, however that's too late for the out of the box props.conf's to take effect. This causes things like the the

"FIELDALIAS-eventName-for-aws-cloudtrail-command = eventName AS commandrename eventName as command"

in props.conf to fail unless I go in and modify it to be event.eventName. I'd like to fix this before it gets to SPL. Is there a way to do this easily? Thanks!

[tscroggins]

Hi @cybersecnutant,

If we assume your events are JSON-formatted:

{"event.field1": "value1", "event.field2": "value2", "event.field3": "value3"}

you can remove the event. field prefix using SEDCMD in props.conf on your heavy forwarder:

# local/props.conf
[your_sourcetype]
SEDCMD-strip-event = s/"event\.([^"]+)"/"\1"/g

or a combination of RULESET in props.conf and a transform in transforms.conf (together, an ingest action) on either your heavy forwarder or your indexer(s):

# local/props.conf
[your_sourcetype]
RULESET-strip-event = strip-event

# local/transforms.conf
[strip-event]
INGEST_EVAL = _raw:=replace(_raw, "\"event\.([^\"]+)\"", "\"\\1\"")

You can also reference the same INGEST_EVAL transform in a TRANSFORMS setting in props.conf. The difference is where in the pipeline the various methods execute. SEDCMD and TRANSFORMS execute between typingQueue and rulesetQueue (along the path for "raw" data), and RULESET executes between rulesetQueue and indexQueue (the injection point for parsed or "cooked" data).

Your final raw event would be:

{"field1": "value1", "field2": "value2", "field3": "value3"}