topic Why does INGEST_EVAL duplicate my events if I assign a value to _raw? in Getting Data In

Why does INGEST_EVAL duplicate my events if I assign a value to _raw?

peter_krammer — Thu, 05 Aug 2021 13:59:24 GMT

When I configure INGEST_EVAL to replace _raw with something else, it duplicates the event.

Splunk Enterprise Version 8.2.1

props.conf:

[source::http:splunk_hec_token]
TRUNCATE = 500000
SHOULD_LINEMERGE = false
KV_MODE = json
TRANSFORMS-fdz_event = fdz_event

transforms.conf

[fdz_event]
INGEST_EVAL = _raw="Test"

Output:

Re: Why does INGEST_EVAL duplicate my events if I assign a value to _raw?

peter_krammer — Thu, 05 Aug 2021 14:50:59 GMT

I found a workaround to circumvent this bug.

tranforms.conf

[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.