topic Re: json messages not always indexed in Getting Data In

What is causing json messages to not always be indexed?

dhuynh — Tue, 13 Jun 2023 13:47:55 GMT

Hi everyone,

For one of our client we are sending in json log data via log4j2 to the splunk cloud HEC token.

we are using the /event/collector/raw endpoint.

What I notice is that the fields are not extracted consistently. We do not see any pattern in our process so we cannot pinpoint the exact location of the issue.

dhuynh_0-1686606054908.png

I am using the following source type with its configs:

dhuynh_2-1686606398965.png

Hopefully can someone see what might cause this issue.

Thankyou in advanced.

Duy

Re: json messages not always indexed

nyc_jason — Tue, 13 Jun 2023 04:15:20 GMT

Hello dhuynh, there are a few possible reasons this could be happening. First, please check for payload character set issues (such as non UTF-8 characters, which can cause JSON to break. Also, check the splunk logs for errors. You can find HEC parsing errors in the _introspection index.

Re: json messages not always indexed

dhuynh — Tue, 13 Jun 2023 11:42:02 GMT

@nyc_jason thankyou for your fast reply.

when checking the _introspection index I dont see any parsing error. Everything gets parsed correctly. what is weird is that it happens randomly. so when I rerun the process again then the data might be parsed correctly.