Getting Data In

ERROR failed to post events "invalid data format": Are these bad?

adammike
New Member

Looks like I have a malformed record in Kafka, I assume that it will keep trying to post the invalid events until the data ages out of the topic in some number of days? I still see data flowing from Kafka into HEC, so I'm not sure if this is a problem or not. I'd like to be sure one way or the other.

Jun 18 10:16:56 a0001p5rlog0003 messages [2019-06-18 12:16:23,464] ERROR failed to post events resp={"text":"Invalid data format","code":6,"invalid-event-number":8,"ackId":7672}, status=400 (com.splunk.hecclient.Indexer:181)
Jun 18 10:16:56 a0001p5rlog0003 messages [2019-06-18 12:16:23,465] INFO add 1 failed batches (com.splunk.kafka.connect.SplunkSinkTask:322)
Jun 18 10:16:56 a0001p5rlog0003 messages [2019-06-18 12:16:23,465] INFO total failed batches 1 (com.splunk.kafka.connect.SplunkSinkTask:47)
Jun 18 10:16:56 a0001p5rlog0003 messages [2019-06-18 12:16:23,465] INFO handled 1 failed batches with 9 events (com.splunk.kafka.connect.SplunkSinkTask:130) 
0 Karma

adammike
New Member

The root cause of this is that HEC is rejecting specific log entries because they are too large. I have opened an additional question to figure out how to fix that.

https://answers.splunk.com/answers/753896/is-it-possible-to-change-the-maxvaluesize-for-hec.html#que...

0 Karma

DavidHourani
Super Champion

Any way to purge these events before they age out ?

0 Karma
Get Updates on the Splunk Community!

Enterprise Security Content Update (ESCU) | New Releases

In January, the Splunk Threat Research Team had one release of new security content via the Splunk ES Content ...

Expert Tips from Splunk Professional Services, Ensuring Compliance, and More New ...

Splunk Lantern is a Splunk customer success center that provides advice from Splunk experts on valuable data ...

Observability Release Update: AI Assistant, AppD + Observability Cloud Integrations & ...

This month’s releases across the Splunk Observability portfolio deliver earlier detection and faster ...