Getting Data In

Why does INGEST_EVAL duplicate my events if I assign a value to _raw?

peter_krammer
Communicator

When I configure INGEST_EVAL to replace _raw with something else, it duplicates the event. 

Splunk Enterprise Version 8.2.1

props.conf:

Spoiler
[source::http:splunk_hec_token]
TRUNCATE = 500000
SHOULD_LINEMERGE = false
KV_MODE = json
TRANSFORMS-fdz_event = fdz_event

transforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test"

Output:

splunk_duplicate.png 

Labels (2)
Tags (3)
0 Karma
1 Solution

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

View solution in original post

0 Karma

peter_krammer
Communicator

I found a workaround to circumvent this bug. 

tranforms.conf

Spoiler
[fdz_event]
INGEST_EVAL = _raw="Test", queue=if(isnull(timestamp),"nullQueue", queue)

Because I notices that one of the duplicates has an indexed field "timestamp::none" and the other does not. With this I am routing only one of the to the nullQueue and keep the other.

0 Karma
Get Updates on the Splunk Community!

Splunk Mobile: Your Brand-New Home Screen

Meet Your New Mobile Hub  Hello Splunk Community!  Staying connected to your data—no matter where you are—is ...

Introducing Value Insights (Beta): Understand the Business Impact your organization ...

Real progress on your strategic priorities starts with knowing the business outcomes your teams are delivering ...

Enterprise Security (ES) Essentials 8.3 is Now GA — Smarter Detections, Faster ...

As of today, Enterprise Security (ES) Essentials 8.3 is now generally available, helping SOC teams simplify ...