Getting Data In

Why does INGEST_EVAL duplicate my events if I assign a value to _raw?

peter_krammer
Communicator

When I configure INGEST_EVAL to replace _raw with something else, it duplicates the event. 

Splunk Enterprise Version 8.2.1

props.conf:

Spoiler
[source::http:splunk_hec_token]
TRUNCATE = 500000
SHOULD_LINEMERGE = false
KV_MODE = json
TRANSFORMS-fdz_event = fdz_event

transforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test"

Output:

splunk_duplicate.png 

Labels (2)
Tags (3)
0 Karma
1 Solution

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

View solution in original post

0 Karma

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

0 Karma
Get Updates on the Splunk Community!

Splunk + ThousandEyes: Correlate frontend, app, and network data to troubleshoot ...

 Are you tired of troubleshooting delays caused by siloed frontend, application, and network data? We've got a ...

Splunk Observability for AI

Don’t miss out on an exciting Tech Talk on Splunk Observability for AI!Discover how Splunk’s agentic AI ...

🔐 Trust at Every Hop: How mTLS in Splunk Enterprise 10.0 Makes Security Simpler

From Idea to Implementation: Why Splunk Built mTLS into Splunk Enterprise 10.0  mTLS wasn’t just a checkbox ...