Getting Data In

Why does INGEST_EVAL duplicate my events if I assign a value to _raw?

peter_krammer
Communicator

When I configure INGEST_EVAL to replace _raw with something else, it duplicates the event. 

Splunk Enterprise Version 8.2.1

props.conf:

Spoiler
[source::http:splunk_hec_token]
TRUNCATE = 500000
SHOULD_LINEMERGE = false
KV_MODE = json
TRANSFORMS-fdz_event = fdz_event

transforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test"

Output:

splunk_duplicate.png 

Labels (2)
Tags (3)
0 Karma
1 Solution

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

View solution in original post

0 Karma

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

0 Karma
Get Updates on the Splunk Community!

Data Management Digest – December 2025

Welcome to the December edition of Data Management Digest! As we continue our journey of data innovation, the ...

Index This | What is broken 80% of the time by February?

December 2025 Edition   Hayyy Splunk Education Enthusiasts and the Eternally Curious!    We’re back with this ...

Unlock Faster Time-to-Value on Edge and Ingest Processor with New SPL2 Pipeline ...

Hello Splunk Community,   We're thrilled to share an exciting update that will help you manage your data more ...