Index Field Transform not Catching Every Event

aaron_gibby — Fri, 05 Mar 2021 21:29:28 GMT

I'm running a simple transform to change the index from "tenable" to "tenable-dc" for one of my sourcetypes.

Props.conf

[tenable:sc:vuln] TRANSFORMS-~dcfilter = dcfilter

Transforms.conf

[dcfilter] REGEX = ([Dd][Cc]01) FORMAT = $0-dc DEST_KEY = _MetaData:Index

The problem that I'm having is that the transform is not catching every event. I have 164 events that the filter should catch, but only 156 events are indexed in the new index (tenable-dc).

If I run the following search command, it catches all 164 events:

index=tenable* sourcetype=tenable:sc:vuln | regex _raw = "([Dd][Cc]01)"

I can't find any similarities between the 8 "missed" events or differences between those events and the 156 "captured" events. My first thought was that the regex was wrong, but the search-time regex works. Does anyone have any experience with index-time extractions missing events?

topic Index Field Transform not Catching Every Event in Getting Data In

Index Field Transform not Catching Every Event